Best LLM Model for coding
Posted by Secret_Scale_492@reddit | LocalLLaMA | View on Reddit | 18 comments
Hey guys I'm trying to setup bolt and looking for a good ollama model which excels in coding. can you suggest a good model that can run in a 3090
Graded_Beast9039@reddit
Qwen 2.5 instruct 32B at quant 5-6
Weary_Long3409@reddit
Using this, quant 8. But how this 32b can be better than 7b-coder tuned version?
Diegam@reddit
yes, the 32b is a lot lot lot better for coding.
Graded_Beast9039@reddit
coder finetunes are definitely better than there corresponding base models but Due to the increased parameters,it is more accurate in first try and also adheres to prompt better and also understands the problem way better
prvncher@reddit
Have you done much testing on the impact of the quant on instruction following?
Graded_Beast9039@reddit
Nah,just going by the basic rule of thumb.if you want the best model you can fit on your machine always going with q4 of larger model
LoafyLemon@reddit
It does not support FIM, I don't think?
Graded_Beast9039@reddit
I'm unaware of FIM.I just mentioned the best gguf he can run on his hardware.what is it if you don't mind me asking?
LoafyLemon@reddit
FIM, or fill in the middle, is a feature that allows code insertion at the current line. Think of it like tab autocomplete, but with spatial awareness for the currently opened file (or files).
brotie@reddit
That’s implemented at the point of inference (the code completion, tool or editor) not the model
Graded_Beast9039@reddit
Or supernova medius for much better interference speeds
XhoniShollaj@reddit
Deepseek is such a great model. Surprised its open sourced.
Healthy-Dingo-5944@reddit
Is the training data open? The definition for opensoure has been overhauled by the open source initiative, here
SatoshiNotMe@reddit
There are a few different products/libs named “bolt”. Which one are you referring to?
Few_Professional6859@reddit
Has anyone used codegeex4:9b-all-q8_0? I find its JavaScript capabilities quite impressive - it helps me solve many problems, and its model size is reasonably compact. Also, I noticed that yi-coder:9b-chat-q4_0 ranks quite high on https://aider.chat/docs/leaderboards/, but I haven't seen any particularly outstanding performance from yi-coder myself.
Narkens@reddit
Someone suggested Codestral-22B-v0.1-IQ3_M in another thread, and Ive been using that one for coding ever since. Mostly for python. Its really good, way better than all other I have tried.
Most of the time it gives me run ready redesigns and code, quite impressive.
Most of the other models didn't quite understand my scripts and what I was doing and suggested crap code with crap design
PermanentLiminality@reddit
I like qwen2.5 coder 7b in Q8. I have 20gb of vram and that allows for some larger context. It can actually do something useful when thousands of lines of code are dropped on to it.
The larger models can be smarter, but if you don't have the spare VRAM, it might not be able to look at your whole project.
Slight-Living-8098@reddit
deepseek-coder