I noticed the following correlations:

- For larger models IQ4 is generally good, but there are exceptions: for example, for Kimi K2 Thinking originally trained using INT4 QAT, Q4_X is the best (as described in the Q4_X recipe by Ubergram: https://huggingface.co/ubergarm/Kimi-K2-Thinking-GGUF )

- Medium size model, from sizes like GLM 4.5 Air to GLM-4.6, I find IQ5 works the best, but this also may have to do something with the fact that they are originally BF16, while larger DeepSeek and Kimi models were FP8 (with K2 Thinking being an exception). GPT-OSS-120B is an obvious exception, since has been post trained in MXFP4 format, so there is no need of higher quant for it.

- Small models like Qwen3 30B-A3B tend to degrade noticeably at IQ4, with IQ5 still possible to notice higher error rate, especially in more complex tasks, with with small models tend to struggle even at the original precision. For vision enabled small models, I find that 8-bit quantizations preserve the quality better.

As of choosing a model to use, for general use I prefer K2 0905 when no thinking is needed or for Roo Code; K2 Thinking could be better, but currently it is not very well supported by Roo Code, and there are still open bug reports related to it. K2 Thinking however works well for other things, including general-purpose chatting in SillyTavern.

Medium size models can be especially useful if fully fit in VRAM. For example, in my case I have just 96 GB VRAM but 1 TB RAM, so while it is enough VRAM to hold even full 256K context cache and common expert tensors of K2 Thinking, it is not enough to fully load GLM-4.6, but I still can fully fit in VRAM something like GLM-4.5 Air or GPT-OSS-120B - this can be useful for bulk processing something that is not too complex, but still too hard for smaller models. Obviously, medium size models can be also useful on RAM-constrained systems, where it is just not practical to use anything bigger.

And finally small models, it is something that I use when I need the best speed for simple tasks. They are not that great for general chat or Roo Code, but being small, it is relatively easy to fine-tune them. For fine tuning, I often use 0.6B-4B range of models, since for specialized simple tasks, fine-tuning something like 30B-A3B or higher is not needed in most cases.