Building agents using SMLs
Posted by shoeshineboy_99@reddit | LocalLLaMA | View on Reddit | 1 comments
If you would want to fine a small language model for a analytical agent. Something which can read docs (text, markdown, json, csv and excel files) and respond to queries which one would you choose?
- Qwen 7bn
- Gemma 9bn
- Phi-4
- llama 3 8bn
- Mistral 12bn
Wally-Gator-1@reddit
- **fresh models** : The models you are mentioning are old to today's standards. Performance have changed dramatically since.
- Small models will not open your excel files on their own unless they have tools to do so. They often hate json. Plus they may hallucinate about your data content.
- To read files, you will need to pass the content as context or embed it in a vector database (RAG method).
- Local models often induce tradeoffs in terms of how much info you can pass (context window) and agentic behavior capabilities.
- Focus on tool usage capabilities (must be agentic enough) and low temperatures (to reduce hallucinations)
For me : Qwen 3 4B or 8B A3B 2507 instruct for local tools with Aider or OpenCode.
I recommend BFCL, artificialanalysis.ai and the Aider Leaderboard to compare models but you will rapidly notice that local models are weak for anything advanced.