Qwen3-30B-A3B runs at 12-15 tokens-per-second on CPU

I got nearly 6 tokens a second running Gemma 3 1b q4_k_m on my PHONE last night! (CPH2083, Oppo A12, 3 GiB RAM, some PowerVR GPU that could get 700 FPS simulating like 300 cubes with a Java port of Bullet Physics in VR. Not exactly amazing these days. Doesn't even have Vulkan support yet! Phone is a *SUPER BUDGETY*, like 150 USD, from 2020. Also by the way, Android 9.) Firefox had worse performance rendering the page than the LLM's own speed LOL. Did take nearly 135 seconds for the first message since my prompts were 800 tokens. I could bake the stuff into the LLM with some finetuning I guess. Never done that unfortunately. (On my 2021 HP Pavilion 15 with a Ryzen 5 5600H, 16 GiB of RAM, and a 4 *GB* VRAM GTX 1650 - mobile, of course, a TU117M GPU - THAT runs this model at 40 tokens a second, and could probably go a lot faster. I did only dump like 24 layers though, funnily enough.) Most fun part is how much this phome struggles with rendering Android apps or running more than one app in the background LOL. There barely is more than 1 *GB* of RAM ever left. And it runs a modern LLM ***fast*** (well, at least inference is fast...!).