Which Version of Qwen 3.6 for M5 Pro 24g
Posted by utnapistim99@reddit | LocalLLaMA | View on Reddit | 3 comments
I have m5 pro with 24GB ram setup. I am not sure to run Q4 version. But i couldn’t find the good Q3 solution. Can you recommend one? I want to try qwen 3.6 with ollama.
KokaOP@reddit
iq4_nl is really good
it was taking ~24gb on my l40s
tmvr@reddit
It's not like you have many options with the default 16GB VRAM allocation there. You will have to go IQ3_XXS or IQ3_S if you want to have some space left for KV and context:
https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF
PapercutsOnPenor@reddit
try and figure out