Getting llm on low end phone
Posted by hemtai_lover@reddit | LocalLLaMA | View on Reddit | 3 comments
So I have samsung f13 64gb storage 4gb ram and an armv7 I have seen a lot of post saying that running llm on armv7 is hard if not pain full but I still want ot try but I don't know where and how to start if Please help
a_beautiful_rhind@reddit
Ideally you use the phone as a client to connect to literally anything else.
LivingLinux@reddit
Perhaps it's possible to run it on the GPU with Google AI Edge Gallery. The choice of models is very limited, but if you can run on the GPU, it might be faster.
https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery
I was surprised it was able to use the Mali-G68 with Hammer 2.1 1.5B q8.
https://youtu.be/bSuh6Ok5tOw
Horus_simplex@reddit
PocketPal with the smallest model possible you'll find on it. It will be very very dumb, but quite funny.