What’s the fastest, smallest, smartest LLM today? (3b or less)

I haven't yet approached that direction of optimization. I'm using the ollama native phi-3 model. I'm getting ~70 tokens per second on a laptop 3060 gpu The verbosity is an issue but I completely solved it through prompt structure. To put it simply, way simpler than what actually is, I tell it to think before answering. No chain of thoughts, just a thought section before an answer section. Then I discard the thoughts from the actual answer. I do still save it for QA reasons but I do see it as mostly useless. I think small models main strength is how many times you can run the prompt simultaneously. Yes it's 70tps, but you can easily run 10 or so concurrent and pool the answers together. You don't really need more than 5 and usually 3 is fine. But I haven't tested it thoroughly yet. (Still figuring out the best database/analytics solution to implement) My personal framework is quite fragile right now but I'm focusing on scaling "horizontally". My main performance bottleneck now is the time to first token. It's very bad (like 1.5s+) right now. Anyways from what I've read, but not tested, 8 bit quantization is the best accuracy/speed trade-off.