Experience with medium sized LLMs

Posted by icarusinvictum@reddit | LocalLLaMA | View on Reddit | 6 comments

I have tried to use several models on my 8gb ram MacBook and concluded that 4b parameters models are just “stupid” for my tasks (i.e. summarisation of pdfs, language learning, etc.).

Online AI services fulfils my needs, however I still want to try implement local ai somehow, maybe you have any ideas?

Models that I tried:

• gemma3:1b

• gemma3:4b

• qwen3:4b

• phi4-mini

• gemma4:e2b