imatrix GGUF for KangalKhan-RawRuby-7B released

Posted by Feztopia@reddit | LocalLLaMA | View on Reddit | 2 comments

mradermacher released imatrix quants for a model which I personally like (I'm still waiting for more mature Llama3 8b models before changing to that, so I let them cook). He seems to be one of the new "TheBloke's". Link to the weighted imatrix quants: [https://huggingface.co/mradermacher/KangalKhan-RawRuby-7B-i1-GGUF](https://huggingface.co/mradermacher/KangalKhan-RawRuby-7B-i1-GGUF) Link to the original model: [https://huggingface.co/Yuma42/KangalKhan-RawRuby-7B](https://huggingface.co/Yuma42/KangalKhan-RawRuby-7B) To the people who prefer bigger models: Im sorry but I only follow "small" large language models. And imatrix quants are very exciting for me in this regard. What I like about KangalKhan-RawRuby: Seems to follow System prompt very well. What I don't like: if you allow it to repeat itself to often, it won't be able to stop the repetition (If you know 7b models without repetition problems, let me know)