Run Llama 3.2 3B on Phone - on iOS & Android

Posted by Ill-Still-6859@reddit | LocalLLaMA | View on Reddit | 132 comments

Hey, like many of you folks, I also couldn't wait to try llama 3.2 on my phone. So added Llama 3.2 3B (Q4_K_M GGUF) to PocketPal's list of default models, as soon as I saw this post that GGUFs are available!

If you’re looking to try out on your phone, here are the download links:

As always, your feedback is super valuable! Feel free to share your thoughts or report any bugs/issues via GitHub: https://github.com/a-ghorbani/PocketPal-feedback/issues

For now, I’ve only added the Q4 variant (q4_k_m) to the list of default models, as the Q8 tends to throttle my phone. I’m still working on a way to either optimize the experience or provide users with a heads-up about potential issues, like insufficient memory. but, if your device can support it (eg have enough mem), you can download the GGUF file and import it as a local model. Just be sure to select the chat template for Llama 3.2 (llama32).