most hackable coding agent

Posted by mnze_brngo_7325@reddit | LocalLLaMA | View on Reddit | 9 comments

I find with local models coding agents need quite a lot of guidance and fail at tasks that are too complex. Also adherence to style and other rules is often not easy to achieve.

I use agents to do planing, requirement engineering, software architecture stuff etc., which is usually very specific to my domain and tailoring low resource LLMs to my use cases is often surprisingly effective. Only missing piece in my agentic chain is the actual coding part. I don't want to reinvent the wheel, when others have figured that out better than I ever could.

Aider seems to be the option closest to what I want. They have python bindings but they also kind of advise against using it.

Any experience and recommendations for integrating coding agents in your own agent workflows?

[-]

Nomski88@reddit

From what I've seen, to do coding successfully via local models you can't go less than Q6 models.

[-]

mnze_brngo_7325@reddit (OP)

My experience is you can. You only have to invest some patience, prompt engineering and lower the task complexity by breaking things down, having checks and control loops. And that is exactly the point of my post. This high level analysis and divide and conquer approach is what I want to control with my own agents and a human in the loop to a degree where the actual coding tasks are simple enough for a small model to handle.

I want to trade compute so that most of it is in the design phase and coding becomes less crucial. Jumping straight into coding was always a bad idea, even before AI.

[-]

TomLucidor@reddit

Then what is your tool belt?

[-]

mnze_brngo_7325@reddit (OP)

If you are asking about agent/workflow frameworks: mostly custom-made abstractions build around low-level API calls, but I'm using pydantic-ai more and more recently, since it's still low-level enough and thus hackable and favors strong typing. It uses pydantic and JSON schema for tools and structured output, which is convenient to build with. Also, its vendor integrations let me get rid of litellm (which sucks), when I build something that needs to be vendor-agnostic.

[-]

MrMisterShin@reddit

I agree… I always go Q8 and also configure settings like temperature.

Some models are more sensitive to quantisation, than others.

[-]