OmniCoder-9B | 9B coding agent fine-tuned on 425K agentic trajectories

[-]

chrisoutwright@reddit

Could not get it to work with GitHub Copilot Chat (MSC Insiders, OpenAI compatible), it would generate the tool call request but then somehow (even when CMD was executed via IDE) Copilot says request failed. Seems like an issue in the chat template?

Reply

[-]

chrisoutwright@reddit

actually, when turning off thinking (which is kind of strange in llama-server .. , need to add this: \--chat-template-kwargs "{\\"enable\_thinking\\":false}" \^ \--reasoning-budget 0 then it works also in github copilot in the MSC Insider. .. seems like a problem with tool calls within thinking tags or so ..

Reply

[-]

phil123456789101113@reddit

trying to serve it with ollama version 0.16.2 I get llama\_model\_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35' llama\_model\_load\_from\_file\_impl: failed to load model time=2026-03-30T00:27:41.808+02:00 level=INFO source=sched.go:448 msg="NewLlamaServer failed" model=c:\\...\\blobs\\sha256-550e8f7253c8e07997fbce2570d37259b69b0d21faf77e5ed518d4ee4c73d8b3 error="unable to load model: c:\\...\\blobs\\sha256-550e8f7253c8e07997fbce2570d37259b69b0d21faf77e5ed518d4ee4c73d8b3" any help appreciated

Reply

[-]

tom_mathews@reddit

Behavioral cloning from Opus traces means the read-before-write recovery is pattern-matched, not reasoned. Curious how it handles novel error states outside the training distribution, that exact failure mode wrecked a prod deploy for us once

Reply

[-]

kayteee1995@reddit

I use Kilo Code to manage AI Agents. Someone has used OmniCoder 9B and Kilo (or Roo) and it really works, please share with me the most optimal and efficient llama-server config (avoid loops and <tool\_call> errors while Thinking).

Reply

[-]

Ueberlord@reddit

unfortunately, I cannot recommend the omnicoder 9b for more complex tasks at the moment. I had it (q8_0 gguf, llama.cpp b8288, temp 0.6, top p 0.95, top k 20) analyze our vue app and asked if it could summarize the API requests executed during usual usage patterns, it failed and got into a loop. exact same prompt given to unsloth Qwen3.5-27B-UD-Q2_K_XL.gguf (same parameters) worked fine on the first try. this is 8.9G omnicoder vs 11G q2_k_xl of unsloth. both can be run on 16G VRAM devices, I would recommend the 27B model to anyone for now.

Reply

[-]

kayteee1995@reddit

Yes! that's true! When working with agentic code (I use Kilo Code to manage agents), 9B can only complete a few simple tasks like simplifying css code, changing attribute of elements,... things that don't require much reasoning. Whenever I use 9B for roles like orchestrator or architect, it becomes "dumb" and "hallucination", asking the same thing over and over again, <tool\_call> while thinking. overall disappointing. However, using it as a chatbot or information analyze assistant (through lmstudio) seems to work better. With larger models like the 35B A3b or 27B, acting as AI agents seems more suitable, the 27B is smarter but slower, while the 35B is fast but lacks depth.

Reply

[-]

kastaldi@reddit

q2 ? I never tried quants under 4, how do these kind of LLMs perform ?

Reply

[-]

Ueberlord@reddit

actually quite well with the Qwen3.5 architecture, you can run a Q3 quant of the 27b model from bartowski with about 80k context size, works very well for me.

Reply

[-]

kastaldi@reddit

Thanks! I thought lowest quants were more of a "niche experiment" than something of real use. I was a fan of Bartowski LLMs, then I discovered Unsloth. I'll give it a try for sure.

Reply

[-]

United-Rush4073@reddit

Thanks for the feedback! Back to the drawing board.

Reply

[-]

jopereira@reddit

I can only take a 16k context on 16Gb, and that's too small for many workloads. A simple task takes forever not because the model is slower (50t/s vs 80t/s with 9B) but because Roo Code does needs to make a lot of context condensing along the way. It's still doing it as I write this although the code is ocmpleted (because it is trying to run it). I guess with 24Gb the 27B model would be the king, but on 16Gb... it needs compromises. But perhaps I'll use 27B for planning and 9B for coding...

Reply

[-]

kayteee1995@reddit

I encountered a situation where the agent kept asking the same question again and again until Kilo code reported a loop error (asking me whether I should decide to read all files at the same time or read each file individually). I use llama-server router-mode (OmniCoder-9B) and Kilo Code as agent manager (Orchestrator mode) mode).

Reply

[-]

Holiday_Purpose_3166@reddit

This is absolutely mental. On my Next.js/Rust suite it beats 27B https://preview.redd.it/v07u9bylpspg1.png?width=2396&format=png&auto=webp&s=df3858ad95bb4d867f50494abfda7a420822a58b

Reply

[-]

sixcommissioner@reddit

looping isnt a sampler problem imo. the model has no external signal that its repeating itself. repeat penalty is a band-aid. if you track tool call frequency over a sliding window the distribution shifts noticeably 5-8 calls before full spiral. easier to kill and retry from checkpoint than tune penalties

Reply

[-]

Deep_Traffic_7873@reddit

Is this model 9b better than qwen3.5 35B-A3B?

Reply

[-]

United-Rush4073@reddit

I wouldn't say so yet, but we plan on fine-tuning that one!

Reply

[-]

sgmv@reddit

Wouldn't the 27B be a better candidate ? As it has been proven superior to 35B ?

Reply

[-]

Uncle___Marty@reddit

qwen 3.5 9B has absolutely turned out to be a master coding agent for its size. I mean, personally I would compare it to trained 100B+ agents right now. While a LOT of attention has been around these low size models I honestly dont think its even close to what people should be shouting about. People hail the big and medium models but we just got a small model that can compete with the medium range and come out with few wounds. If anyone at the qwen team ever reads this, thank you. Small models are the future and I dont care how much I get down voted but local models should be small and powerful. Qwen is that model. Underestimate qwen 3.5 9B and you're an idiot. This is THE next level of small models right now. DO NOT underestimate it if you're trying to find a solution. It might not work for you but think of it like a 100B model in terms of what it can do, and NOT its world knowledge (which is amazing for its size but 9B dude).

Reply

[-]

ghulamalchik@reddit

> DO NOT underestimate it Ok ok, I won't. Sheesh.

Reply

[-]

Borkato@reddit

I am constantly blown away at the quality of 3.5 35B-A3B. A few more generations with this kind of improvement and we’ll be at current sonnet level locally.

Reply

[-]

sonicnerd14@reddit

Moe models like qwen3.5 35b, GLM 4.7 flash, or gpt oss are magic for local. Especially qwen3.5 moe models since they come native with vision. I've been playing around with my 2 machines, one that has 16gb vram and 32gb of ram, and one with 8gb vram and 48gb of ram. When I learned about how much faster performance qwen3.5 35b got moe cpu offloading + full gpu offload, it lead me to experiment with my 8gb system and also the other models on both. It's crazy how such tweaks now gives even my desktop system with the 8gb of vram useable speeds with such capable models. The laptop on the other hand is blazing fast, with GLM 4.7 flash beating qwen3.5 in speed in most cases and in coding. It's clear the direction for local should be more moe multimodal models like qwen3.5. If the efficiency increases with the intelligence at this rate, then we likely won't need frontier nearly as much as we used too.

Reply

[-]

Eyelbee@reddit

What's so special about 35BA3? Isn't 27B just literally better? Are people just using 35B for throughput?

Reply

[-]

Borkato@reddit

Wait GLM 4.7 flash beats qwen 3.5 in coding?

Reply

[-]

sonicnerd14@reddit

From my tests, it appears to do so in most cases. Could experiment with increasing experts with qwen3.5 and making it better than GLM 4.7 flash overall.

Reply

[-]

Borkato@reddit

Wait now I’m confused, you can increase experts?!

Reply

[-]

sonicnerd14@reddit

Yes, you can with moe's. At least in LMStudio you can, you probably can in Llama.cpp, but I dont't know the exact command for that. You can increase the experts, which is essentially adjusting the number of active parameters that are being used by the model.

Reply

[-]

Deep_Traffic_7873@reddit

For me glm4.7-flash is slower than qwen3.5 35b a3b which quant and optimization did you use?

Reply

[-]

sonicnerd14@reddit

Q4\_K\_S for GLM 4.7 Flash REAP 23B A3B Absolute Heresy I1 on my laptop, and Q4\_K\_M for it on my desktop. The REAP compressed models have virtually very little difference in quality compared to the full 30b quants. Give it a try and see what you get.

Reply

[-]

Serious-Log7550@reddit

I have similliar setup 4060 8gb + 32Gb DDR5, could you provide yours llama-server run string with cpu moe offloading?

Reply

[-]

sonicnerd14@reddit

I'm mostly using LMStudio right now, I have LLama.cpp but haven't tried it out yet. Just make sure you offload your GPU layers to the max, and then for your system you can try something like --n-cpu-moe 24 or if you want to play around with how fast you can get your gens somewhere in between that and --n-cpu-moe 34 is probably where you want to aim.

Reply

[-]

Subject-Tea-5253@reddit

I have a similar setup: RTX 4070 8GB + 32GB of RAM. Here is the command I use ```bash llama-server \ --model /home/imad-saddik/.cache/llama.cpp/Qwen3.5-35B-A3B-Q4_K_M.gguf \ --ctx-size 128000 \ --fit 1 \ --flash-attn 1 \ --threads 6 \ --no-mmap \ --jinja \ --cache-type-k q8_0 \ --cache-type-v q8_0 \ --chat-template-kwargs "{\"enable_thinking\": false}" \ --parallel 1 \ --port 8088 ``` I get around `33t/s` when the conversation is still small, and it drops to around `26t/s` when I reach 100k+ tokens.

Reply

[-]

AlwaysLateToThaParty@reddit

The vision is killer for qwen. cut/paste - "give me a list of those files"

Reply

[-]

ambassadortim@reddit

Unfortunately idk if it'll be from the qwen family

Reply

[-]

MaCl0wSt@reddit

the near future is indeed exciting. It's hard to imagine current-Opus-level intelligence and world knowledge from a raw >40B model, but I do think we can get close to that with the appropriate orchestration and frameworks and like you said, a couple more gens. Early 2026 has been amazing already in this regard, here's hoping we'll read back at this by December and say "oh man, if they knew"

Reply

[-]

tat_tvam_asshole@reddit

idk, it didn't work so well in my testing, kept getting stuck in loops trying to resolve packages and continually flipflopping the same solutions back and forth. also tried building a simple codebase of agent skills with sonnet 4.6 as the senior dev reviewing and directing it, and it just couldn't perform. 27B on the other hand is decent.

Reply

[-]

AlwaysLateToThaParty@reddit

I genuinely think it relates to coding styles, and whether yours are aligned with the test material. People program in an infinite number of ways.

Reply

[-]

tat_tvam_asshole@reddit

having an agent write their own code and screwing up the basic package imports is pretty mindblowingly bad

Reply

[-]

IrisColt@reddit

We would be grateful if you’d provide the language, use case, and tools the agent used... it’ll help us dig deeper.

Reply

[-]

tat_tvam_asshole@reddit

talking about Qwen3.5-9b

Reply

[-]

snmnky9490@reddit

That is not the language, use case, or tools that the agent used lol

Reply

[-]

tat_tvam_asshole@reddit

I believe he's refering to the Omnicoder-9b not Qwen. In any case, 27B is much better than 9B anyway.

Reply

[-]

FUS3N@reddit

I feel like people should give attention to small models so you know researchers focus on improving them more so there is a time where models like these genuinely do crazy good on everything not on some specific tests, which imo is ideal scenario where a 9b does genuinely better than a 30b on everything, smaller better and faster

Reply

[-]

IrisColt@reddit

I'd appreciate if you could tell me the language, the use case, and the tools the agent used. Just to derive further insights...

Reply

[-]

PaceZealousideal6091@reddit

Doesn't benchmarks show it inferior to 35B moe model? Do you have a different experience?

Reply

[-]

jtonl@reddit

Benchmark =/= Usage

Reply

[-]

AlwaysLateToThaParty@reddit

This is increasingly going to be the case as models get more capable. They'll specialise, and not just in the way intended when being built. They'll align with different people in different ways. This is one of the core reasons why local models are the only thing that matters to me; consistency. I can't have the software supplier changing model configurations, no matter how good a reason you think you have for doing it. I use it in production. We can't have your changes fucking up our things. Pretty much applies to every use case.

Reply

[-]

RestaurantHefty322@reddit

The read-before-write pattern alone makes this worth trying. That's the single biggest failure mode we hit with smaller models in agentic loops - they just start writing code without checking what's already there. Ends up clobbering imports, duplicating functions, the usual mess. We run a setup where background agents handle file exploration and code edits while a heavier model orchestrates. Tried swapping the background agents from a 70B to Qwen3.5-9B last week and honestly the gap was smaller than expected for most tasks. The place where it fell apart was multi-step error recovery - the 9B would fix the immediate error but miss the upstream cause. If OmniCoder genuinely learned those recovery patterns from the Opus/GPT-5 traces, that could close the gap for real workloads. One thing to watch: 425K trajectories sounds like a lot but the distribution matters more than the count. If most of those traces are Python web dev (which training sets tend to skew toward), performance on infra code or less common languages might not hold up.

Reply

[-]

IrisColt@reddit

>One thing to watch: 425K trajectories sounds like a lot but the distribution matters more than the count. You nailed it... I don't expect my pet niche languages (8086 assembly, Ren'Py, Inform 6/7, Haskell, Cisco IOS, Matlab...) to be well represented, heh

Reply

[-]

lizerome@reddit

To be fair, that's something large models tend to suck at too. The last time I tried writing AMPL/GMPL code with Claude, it couldn't even get the syntax right and constantly hallucinated features which did not exist. Some languages are simply too obscure to be represented in the training data, even at the trillion parameter scale. The upside is that small models are relatively inexpensive to finetune, so if you're serious about your use case, you could easily create a "Qwen-3.5-9B-Haskell" by scraping together examples from RosettaCode/StackOverflow/etc.

Reply

[-]

United-Rush4073@reddit

If you can help me source the data, I can add it to the next version of omnicoder

Reply

[-]

lizerome@reddit

Haskell/ASM (docs, GitHub, distillation from large models, synthetic data checked with a compiler) and Ren'Py (decompiled commercial games) are probably the easiest on that list, the others I wouldn't even know how to approach sadly. Ren'Py is probably the most doable with the least effort actually, since it's effectively a superset of Python (which models are already overtrained on), and every game ever released is source-available by necessity.

Reply

[-]

IrisColt@reddit

>To be fair, that's something large models tend to suck at too. Yeah, Ren'Py is especially tricky for them...

Reply

[-]

RestaurantHefty322@reddit

Yeah the long tail languages are always the first casualty. 425K trajectories probably covers Python/JS/Java heavily and then drops off a cliff. For something like Ren'Py or ZX Spectrum assembly you'd realistically need a dedicated fine-tune on whatever small corpus exists. The general coding ability might still transfer for reasoning through problems but the actual syntax generation will be rough.

Reply

[-]

United-Rush4073@reddit

Big labs just buy ready made datasets for this

Reply

[-]

United-Rush4073@reddit

If you can help me source the data, I can add it into the next version of omnicoder!

Reply

[-]

acetaminophenpt@reddit

Nice stack!

Reply

[-]

JockY@reddit

> ZX Spectrum assembly You gotta be in the UK and probably cut your teeth on 6502, possibly even Z80!

Reply

[-]

IrisColt@reddit

¬‿¬

Reply

[-]

cenanozen@reddit

This guy niches

Reply

[-]

IrisColt@reddit

I hate to admit it, but my tech life is a mess... :-(

Reply

[-]

United-Rush4073@reddit

Thanks for the feedback! We'll add in more distribution next time around from real github prs on popular projects across frameworks / languages.

Reply

[-]

do_u_think_im_spooky@reddit

Tested OmniCoder-9B Q8 against Qwen3-Coder-30B-A3B (MXFP4) on 2x RTX 5060 Ti 16GB. | | OmniCoder-9B (Q8) | Qwen3-Coder-30B (MXFP4) | | ----------- | ----------------- | ----------------------- | | Prompt eval | 903 tok/s | 317 tok/s | | Generation | 36 tok/s | 78 tok/s | 30B MoE is faster on generation (only ~3B active params vs 9B dense), but OmniCoder chews through prompts nearly 3x faster. Gave both the same FastAPI refactoring task asking for diffs. OmniCoder gave a clean single diff with solid explanations. Qwen3-Coder duplicated the entire diff block and used sync Session instead of AsyncSession. Both caught all the bugs though. For a 9B fine-tune matching a 30B MoE on output quality, the agent trace training is clearly pulling its weight. Both fit in 32GB VRAM comfortably — OmniCoder Q8 with full 262k context only uses ~20GB.

Reply

[-]

United-Rush4073@reddit

Thanks for the feedback!!

Reply

[-]

Odd-Ordinary-5922@reddit

So many things wrong with this... you are using mxfp4 for a model that wasnt post trained on mxfp4 and you are using qwen3 coder 30b a3b and not the newer qwen3.5 35b a3b. Obviously the newer one will be better than a model that is 7 months old.

Reply

[-]

do_u_think_im_spooky@reddit

Fair point on the MXFP4. Had mainly been using that quant for the speed increase on blackwell architecture. Swapped some MXFP4 quants out for Q4_K_XL The reason I used Qwen3-Coder-30B over Qwen3.5-35B is that it's a coding-specific model, comparing a coding finetune to a general model isn't really the point. That said, tested the 35B anyway with the same FastAPI refactoring task: | model | PP (t/s) | TG (t/s) | | ----------------------- | -------- | -------- | | OmniCoder-9B Q8 | 3076 | 38.9 | | Qwen3.5-35B-A3B Q4_K_XL | 2297 | 61.2 | 35B gave a clean diff, no duplication. Better than the 30B in the original post. Still mixed async routes with sync Session though, same mistake. OmniCoder handled that correctly. For a general model it did well, but the coding-specific training is showing where it matters.

Reply

[-]

Odd-Ordinary-5922@reddit

ah nice and sorry for my previous reply It was a bit aggressive

Reply

[-]

Tasio_@reddit

Thanks for sharing, I was looking for a quick comparison of this two models, OmniCoder-9B seems worth trying.

Reply

[-]

do_u_think_im_spooky@reddit

I was just testing the coding model against the Omni model which is designed for coding also. Not the 35B model which isn't a coding specific model

Reply

[-]

mecshades@reddit

Curious about your comment about "asking for diffs." Does OmniCoder produce git patches instead of rewriting entire source files? If so, that's absolutely insane and I want to learn how you've achieved it. I've had little success asking Qwen3-Coder-Next for patches- they always come out broken.

Reply

[-]

do_u_think_im_spooky@reddit

The benchmark task explicitly asked for unified diffs rather than full rewrites. Just prompt it that way and OmniCoder handles it cleanly. The agent trace training is probably why, it's seen a lot of real coding agent output which tends to use diff format natively. I didn't verify git apply compatibility directly so can't promise that, but the format was clean with no duplication.

Reply

[-]

Deep_Traffic_7873@reddit

Is omnicoder 9b better than qwen3.5 35b a3b?

Reply

[-]

do_u_think_im_spooky@reddit

On actual coding tasks OmniCoder is still ahead, the 35B is a better all-rounder but not purpose-built for code.

Reply

[-]

vk3r@reddit

A question. Is the GGFU format compatible with Vision's mmproj?

Reply

[-]

esuil@reddit

It is, and this fine-tune appears to work with original mmproj. But since it is not included, I can't vouch for deterioration of quality. But it does work with it, from what I tested.

Reply

[-]

United-Rush4073@reddit

Sorry, we just forgot to include it but if u use the default qwen files it should be fine. I removed it during training.

Reply

[-]

AlwaysLateToThaParty@reddit

Yep. I run it on my work computer.

Reply

[-]

PaceZealousideal6091@reddit

How does it compare to Qwen 3.5 35B ? Any comparitive benchmarks with it? Any idea if they plan to make the OmniCoder 35b moe?

Reply

[-]

United-Rush4073@reddit

Still working on finding compute to finetune the big one!

Reply

[-]

W1k0_o@reddit

Played around with this model for a couple hours it made tons of mistakes writing simple html/javascript. Maybe I'm doing something wrong or misusing the model but I don't see what all the hubbub is about just seems mediocre to me.

Reply

[-]

United-Rush4073@reddit

Thanks for the feedback!

Reply

[-]

Tasio_@reddit

After reading the positive comments, I decided to try it out (Q8), but it didn’t work well for my use case. I asked a few questions about some complex code and it gave incorrect answers. I also tried using the same prompt I use to test other models, where I ask it to build a blog with specific characteristics. Although it generated the project, it wasn’t working. I then asked it to run the end-to-end tests and fix the issues until everything worked, but after about two hours I gave up. For now, I’m going to continue using `unsloth_Qwen3.5-35B-A3B-GGUF_Qwen3.5-35B-A3B-Q4_K_M`. It may not be as good as the larger models but for my use cases it performs better than Omnicoder9b. It gave me a better answer to the earlier coding question and is able to build a working blog using the same prompt.

Reply

[-]

hurdurdur7@reddit

At which quant? I have found the smaller qwens all been flawed under q6_k, at least for my purposes.

Reply

[-]

W1k0_o@reddit

Q6_K using the recommended reasoning parameters from qwen. However I'm pretty sure I messed up the context and response size. Will give it another fair shake later.

Reply

[-]

TomatilloPutrid3939@reddit

This seems gold. Excited to test. And exited to a 27B version

Reply

[-]

United-Rush4073@reddit

Otw! We're just hunting for more compute!

Reply

[-]

Outdatedm3m3s@reddit

Is there a larger version of this?

Reply

[-]

United-Rush4073@reddit

Otw! We're just hunting for more compute!

Reply

[-]

pilibitti@reddit

very very good. it just one shotted an agentic task that requires 20+ tool calls that Qwen3.5 9B failed despite detailed system prompts (with a blank system prompt no less).

Reply

[-]

United-Rush4073@reddit

Thanks for the feedback! I'm glad you like it!

Reply

[-]

jopereira@reddit

Prompt: "Make a HTML web UI to calculate the first n primes. Use the fastest method available. Option to select n: 100, 1000, 10000 (default), 100000, 1000000 primes. Two panes: left one with buttons, information and progress, on the right one pane to output the numbers. Button to start generation Button to clear results A gauge (full 360º) that shows progress (starting at 12o'clock), including the progress % inside the gauge Make the web UI with elegant color schemes, simple yet modern, responsive and with light/dark modes (dark is default) . Numbers pane can be a scrollable window but the whole UI must be contained in one 16:9 page." Roo Code (VS Code), LM Studio. LEFT: QWEN3.5 27B Q2\_K\_XL (very slow as it had to compress the 16K context several times along the way). RIGHT: Omnicoder 9B Q8 after a 3-4 iterations, but very fast, good dynamic (the reds and export CSV were a post request). 165K context window. https://preview.redd.it/8izv6mugk1pg1.png?width=2427&format=png&auto=webp&s=c0523060e59766bade22ec8bcbdf1711e290e39a

Reply

[-]

docybo@reddit

genuinely impressive work, but worth flagging... training on Claude Opus 4.6 and GPT-5 outputs is explicitly against Anthropic's and OpenAI's ToS. not throwing shade, the model clearly shows results, just surprised nobody's talking about the legal exposure here. dataset release might be a complicated conversation for that reason too

Reply

[-]

jopereira@reddit

Every AI company trained their own models agains ToS of everyone else (redit, facebook, YT, etc.). So, fair enough for me.

Reply

[-]

theowlinspace@reddit

I don't think they care though, and it's extremely unlikely you're going to get into legal trouble over breaking a ToS (The worst they'll do is just deny you service). Keep in mind that their TOS should be respected just as much as how they respected the data they stole for training, and that the legal system has done nothing to that far worse offence.

Reply

[-]

Weird_Search_4723@reddit

u/DarkArtsMastery any plan to release the data? i'm also thinking of doing something similar but don't have the budget to generate these many traces

Reply

[-]

Eyelbee@reddit

Can you make the 27B version of this one?

Reply

[-]

PattF@reddit

This works really really well but runs super slow via LM Studio into Claude Code on my M4 Pro. We're talking like 30 minutes to build an index.html with a basic script.js and styles.css

Reply

[-]

computehungry@reddit

Although I haven't tried it on mac, my guess from my experience on win/linux would be 1) It's a new model and I've seen a lot bugs/unimplemented features with it, including prompt caching (which greatly reduces needed calculations). Might have to wait a while until they sort everything out especially since you're on mac. 2) LM studio might also be the culprit, if your memory isn't being maxed out. It doesn't expose the ubatch flag in llama.cpp (which it runs under the hood) which, after some tuning, 5x'ed my prompt processing speed from LM Studio. CC has a huge system prompt. llama.cpp takes some time to learn and run but it might be worth looking into.

Reply

[-]

mecshades@reddit

I can vouch for this. I asked Qwen3.5 9B Q4\_K\_M to build me a Python MCP server without any additional dependencies and with only one tool that can execute shell commands. I then run the MCP server and I tell llama.cpp's llama-server web UI to talk with it. I now have a coding agent I can talk to directly. It reads files, writes & edits them, is able to daemonize web servers with PM2, and make curl requests to examine the output of the web server. No need for Claude Code, no need for OpenClaw or "AnyClaw" for that matter. I prefer how simple llama.cpp's llama-server web UI is and it doesn't require any additional software to use aside from a web browser.

Reply

[-]

alfpacino2020@reddit

podrias esplicar como lo as escho no logro que la interface web de llama server que tiene ahora la parte de mcp pero no entiendo como cargarlos como que pide usar una web no .json o mediante un localhost

Reply

[-]

Odd-Ordinary-5922@reddit

how many t/s

Reply

[-]

AlwaysLateToThaParty@reddit

Apparently a recent update of llama.cpp related to qwen models increased performance significantly.

Reply

[-]

Shifty_13@reddit

I am new here. I use llama.cpp and ik_llama. What software do you guys use for coding with this model? I am kinda tired of copy-pasting the code... Another question, I see "tools" mentioned a lot, with which software I can play with this functionality?

Reply

[-]

PaceZealousideal6091@reddit

Google a bit about using ide vs code with extensions like cline or kilo code. There are a lot of youtube videos around showing how to use it. Since u use llama cpp, u already know how to expose the oai URL. U can put it into the extension and start using it directly. You may need to use mcps for advanced features like web search etc

Reply

[-]

Shifty_13@reddit

Thanks. Do you have thoughts on opencode? To be used with Cursor, Windsurf, VSCodium? (I am not familiar with these names btw :p ) As you can already tell I am somewhat new to programming. Just trying to find the current best option for local AI enthusiasts. Ideally I would like you use something that is being actively developed on github. I like cutting edge functionality.

Reply

[-]

jopereira@reddit

I'm using Roo Code (I also have Cline and Kilo Code). With RTX5070ti 16Gb, without optimizations, LM Studio does \~70t/s. Will try with llama.cpp This model is a beast! With the prompt bellow, it does not get it right the first time nor did Kat Code Pro nor MinMax M2.5. But correcting errors was a breeze and fast as hell. As fast (faster?) as I remember Grok Code Fast 1 when I had in Cline (as free tier). "Make a HTML web UI to calculate the first n primes. Use the fastest method available. Option to select n: 100, 1000, 10000 (default), 100000, 1000000 primes. Two panes: left one with buttons, information and progress, on the right one pane to output the numbers. Button to start generation Button to clear results A gauge (full 360º) that shows progress (starting at 12o'clock), including the progress % inside the gauge Make the web UI with elegant color schemes, simple yet modern, responsive and with light/dark modes (dark is default) . Numbers pane can be a scrollable window but the whole UI must be contained in one 16:9 page. Put the files/files "AUXILIAR" folder (create it)." Partial screenshot of the plan: https://preview.redd.it/6wbtvdzdbvog1.png?width=578&format=png&auto=webp&s=227631a75245f490cfedb5bd9b090783fd9ce1f2

Reply

[-]

INT_21h@reddit

For people who are *not* experiencing tons of model looping with this, can you please say which quant and sampler settings you're using? I'm using Bartowski's IQ4_NL, the recommended settings - --temp 0.6 - --top-p 0.95 - --top-k 20 - --presence-penalty 1 and an extra - --repeat-penalty 1.0 but I'm still having to watch it like a hawk to ensure it doesn't get stuck in any loops

Reply

[-]

sine120@reddit

Are there any good 3.5-27B or 35B-A3B finetunes with similar results that people have tried and confirmed better? I know there's the Opus-Reasoning distills but I haven't heard anyone who's actually used them much yet.

Reply

[-]

anonynousasdfg@reddit

@HauhauCS if you are reading this, could you please abliterate it with your aggressive method? :)

Reply

[-]

alitadrakes@reddit

New to this, can i run this in LMStudio?

Reply

[-]

saamQ@reddit

noob here. How do I actually use this in an IDE? So far ive setup ollama and one llm, i have no idea about a proper local dev environment tech stack

Reply

[-]

Jaded_Towel3351@reddit

They have a GGUF version, you can use it with llamacpp + Claude code in vscode, unsloth has a tutorial on this, just follow their qwen3.5 tutorial.

Reply

[-]

saamQ@reddit

Can local LLMs work with MCPs? Does VS code + CC do diffs like Cursor?

Reply

[-]

Jaded_Towel3351@reddit

it works just like any paid API or coding agent, if you are talking about showing the difference before and after edit, yes claude code and show that and it can rewind also, but personally i prefer vscode copilot in showing the diffs and comparison, but somehow it only support ollama for local LLM so i have to stick to claude code. If you prefer cursor you can probably swap the paid API to the local api generated by llamacpp also, something like http://locahost:8080/v1.

Reply

[-]

-_Apollo-_@reddit

Copilot chat on vscode supports lmstudio through the oai extension so it should support your solution too no?

Reply

[-]

Jaded_Towel3351@reddit

Just knew about this and it works perfectly! Thanks for the info.

Reply

[-]

-_Apollo-_@reddit

Welcome :)

Reply

[-]

saamQ@reddit

thanks!

Reply

[-]

AlwaysLateToThaParty@reddit

llama.cpp is the OG. The web server (llama-server) exposes an OpenAI format API end point. You configure your tool to connect to that server address, and it uses the model that is loaded with llama-server.

Reply

[-]

saamQ@reddit

Can local LLMs work with MCPs? Does VS code + CC do diffs like Cursor?

Reply

[-]

Comrade_Mugabe@reddit

Building on the above comments, you can also use llama_cpp to host a `llama-server` which will give you a local URL `http://localhost:8080/` (or w/e port you selected), which you can then plug in Roo Code, a VS Code extension. You can host a server with other applications, such as LM Studio, which you could argue is slightly easier. I've just found llama_cpp way superior in performance, especially on my machine.

Reply

[-]

Single_Ring4886@reddit

JESUS DONT TRAIN IT ON GPT "5" !!!!

Reply

[-]

Embarrassed_Adagio28@reddit

Downloading as we speak to test with opencode on a 5070 ti! Looks awesome.

Reply

[-]

Naive_Area6965@reddit

How was it? Is it as good as Claude? (I'm beginner at this)

Reply

[-]

Potential-Leg-639@reddit

Nowhere near claude of course

Reply

[-]

oxygen_addiction@reddit

Lol. No. Claude is probably over 300b parameters and SOTA. Nothing comes close in terms of Open Weight models outside of GLM5/Kimi2.5, and even those are a generation behind.

Reply

[-]

nebulaidigital@reddit

OmniCoder-9B being trained on 425k agentic coding trajectories is interesting mostly because it shifts the benchmark from “writes good code” to “behaves like a tool-using engineer.” The read-before-write and minimal-diff habits matter a lot in real repos, and they’re exactly what most open models still mess up under pressure. I’d love to see a breakdown of where the gains come from: hybrid architecture vs the trace curation vs the scaffolding patterns (Claude Code/OpenCode/Codex-style). Also curious how it handles long-running tasks: does it degrade gracefully when tools fail, or does it spiral? Any evals on real PR-style workflows?

Reply

[-]

Undici77@reddit

Great Job: when I'll try in my daily dev job and I give you a feedback. Currentry I'm using QWEN-CODER models and they are very good. About your project, can you share the entire process from how you distill \`425K agentic trajectories\` to the fine-tune procedure?

Reply

[-]

HeadAcanthisitta7390@reddit

FINALLY NOT AI SLOP mind if i write about it on [ijustvibecodedthis.com](http://ijustvibecodedthis.com) ? cos this is fricking awesome

Reply

[-]

mintybadgerme@reddit

Any idea why I'm getting the dreaded "Failed to load the model. No LM Runtime found for model format 'gguf'!" message on LMStudio? I've updated to the latest beta of LMStudio.

Reply

[-]

Cofound-app@reddit

the fact that a 9B fine tune trained on frontier agent traces can even come close to matching bigger models is kinda wild tbh. we swapped our background coding agent from a 70B to qwen 3.5 9B last week and the gap was way smaller than expected for most tasks

Reply

[-]

Skyne98@reddit

Will you be willing to release the dataset?

Reply

[-]

Serious-Log7550@reddit

It's just a piece of art! It's possible to have Unsloth quants?

Reply

[-]

WithoutReason1729@reddit

Your post is getting popular and we just featured it on our Discord! [Come check it out!](https://discord.gg/PgFhZ8cnWW) You've also been given a special flair for your contribution. We appreciate your post! *I am a bot and this action was performed automatically.*

Reply

[-]

x1250@reddit

Wow this model is really good. Thanks.

Reply

[-]

DevilaN82@reddit

Od this supposed to be used with aider / roocode? Or there is some other setup to test it?

Reply

[-]

Kilithi@reddit

Very cool. Trying it out with OpenClaw to see if it can replace Qwen3.5:9b. I did run into an issue where it says Tools not supported tho.

Reply

[-]

FrogsJumpFromPussy@reddit

Please train a 4b version as well 🥲

Reply

[-]

Lost-Garage-4358@reddit

Raw parameter count matters less than the training recipe and data quality. We've seen 30-40B models punch way above their weight when the RL objectives are well-tuned.

Reply

[-]

Varmez@reddit

Anyone tried this for working on N8N workflows by chance?

Reply

[-]

LoveGratitudeBliss@reddit

Very interesting indeed , any chance of a mlx mac version ? Sounds amazing 👏

Reply

[-]

pmttyji@reddit

There are already [https://huggingface.co/models?other=base\_model:quantized:Tesslate/OmniCoder-9B](https://huggingface.co/models?other=base_model:quantized:Tesslate/OmniCoder-9B)

Reply

[-]

Iory1998@reddit

Has anyone tried this model? How does it fare in your tests?

Reply

[-]

XYSkywalker@reddit

Honestly the most interesting part here isn’t that it’s another coding model — it’s how it was trained. 425k agentic trajectories is basically distilling how frontier models actually work through real tasks: reading files, reacting to diagnostics, editing diffs, retrying after errors. That’s closer to “learning the workflow of a developer” than just predicting the next token in code. If this trend continues, I think the big shift won’t be bigger models, but small models that behave like competent agents. A 9B model that knows how to read → reason → edit → retry might be far more useful in practice than a huge model that just spits out code blocks. The real question is whether this kind of trajectory training scales — because if it does, the next generation of local dev agents could get surprisingly good without needing 100B+ models.

Reply to Post

146 Comments