local llm for macbook air?
Posted by thebrokebuilder@reddit | LocalLLaMA | View on Reddit | 9 comments
I'm thinking of building a mac app that will use a local llm to do content generation and I would like to find a local llm that would work on not so powerful laptops, like the macbook air.
What are your suggestions? So far, from multiple conversations with our group of friends (ChatGPT, Claude, all those guys) the best bet is on llama 3.2 1b quantized. Has anyone run this locally? Curious of what the output would be.
o0genesis0o@reddit
Qwen3 4B instruct 2507? I run that on my macbook pro M1. Subjectively, it feels more solid than the Llama 8b I used to run a few years ago.
Miserable-Dare5090@reddit
I think your iphone has more ram/is able to run bigger models, ironically
wysiatilmao@reddit
Running a local LLM on a MacBook Air is tough due to its limited resources. Look into options like TinyLlama or GPT4All which are designed for low-power devices. These models are efficient and you might find success using them for lightweight tasks locally. If you haven't yet, explore BiteSizedLLMs for more insights on small-scale implementations.
AppearanceHeavy6724@reddit
What you want to accomplish? Chatting, coding, storytelling? My personal take the best allrounders are old but still used models like Llama 3.1 8b and Mistral Nemo - they are still popular for a reason.
Do not run anything below 4B quantized, small quantized models suck ass.
DistanceSolar1449@reddit
Llama 3.2 1b is trash
AppearanceHeavy6724@reddit
It is good among 1B's, but 1B is trash yes.
Lost-Blanket@reddit
What Chip and how much ram?
pulse77@reddit
llama was best last year; this year it is Qwen3; if possible use Qwen3-30B-A3B
souljorje@reddit
First of all use MLX versions of models, you can download and run it easily with LMStudio or use mlx-lm if you can code and wanna save some memory.
Models:
Good luck!