since people were impressed with Bonsai running on WebGPU, how about Gemma?
Posted by douchebanner@reddit | LocalLLaMA | View on Reddit | 1 comments
Posted by douchebanner@reddit | LocalLLaMA | View on Reddit | 1 comments
Tall-Ad-7742@reddit
Gemma is really great, and yes, that can work too, but the amazing thing about bonsai is that, for example the 1B version is only i think 0.3b or 300 MB big, and their 8B model is just, I think, 1 or 2 GB big, while keeping mostly the accuracy (at least I heard)
but still nice project 👍