Yea I guess it depends on the situation, like, if it is just for casual writing type of stuff, I am pretty patient (probably much more than most people) since I enjoy reading what it writes for a scene or a crazy situation I come up with, and then comparing what one model writes compares to a different model or a different fine-tune, or even having it try multiple times, if temperature is somewhat high, to see the different ways it writes about the same situation, so I don't mind waiting 5 or 10 minutes even, or maybe longer, if I think it will be interesting or funny, plus also I can start reading it while it is still midway through writing it, if it is something quite long that it is writing. I mean, if I'm working on something while it is doing its thing, like if I'm doing some work or something, or if I am watching tv or reading a book or something, then, the time it takes doesn't matter to me much if it is 5 seconds or 5-10 minutes I guess. Since I am distracted with whatever else I am doing and can go continue with it whenever I want, so I don't usually get impatient with its speed, even if it is slow. I'm sure I would have my limits though of course, like if it was Llama 405b dense and going from NVMe cpu inference at like 0.01 tokens per second or something crazy, then that would be way too slow. But if the total time is 10 mins or less, or especially 5 minutes or less, per reply, then I don't usually mind too much, and just want what gives the highest quality or most interesting replies.

But, also depends what mood I am in or if I am busy with other things at the time, or what the exact use-case scenario is, or so on.

But yea, luckily Gemma4 26b MoE exists, so, that one is still quite strong even as an MoE. You should definitely try it and compare it a few times with the same prompts to both it and the 31b dense model and see if it seems strong enough, and if it is, then you get to use it at much higher speed than the 31b. And it is very nice since up until now, it wasn't like that with similarly sized MoE models to a similar sized dense model, they used to be MUCH much less smart and less good at writing, so, yea, it was a major breakthrough from Google to make 26b a4b so strong at writing. Still not sure how they did it. When I saw it, it reminded me of back a few months earlier when I saw the Elo scores of the Gemini Flash models compared to Gemini Pro, on LM Arena, how the Flash models barely even had lower Elo scores than the full sized Pro models, which was pretty crazy as well. So I guess it makes sense that they were able to use whatever breakthroughs they made to be able to do that with their Gemini Flash models to also make their Gemma4 26b a4b model so good at writing for a small MoE model.