imatrix GGUF for KangalKhan-RawRuby-7B released
Posted by Feztopia@reddit | LocalLLaMA | View on Reddit | 2 comments
mradermacher released imatrix quants for a model which I personally like (I'm still waiting for more mature Llama3 8b models before changing to that, so I let them cook). He seems to be one of the new "TheBloke's".
Link to the weighted imatrix quants: [https://huggingface.co/mradermacher/KangalKhan-RawRuby-7B-i1-GGUF](https://huggingface.co/mradermacher/KangalKhan-RawRuby-7B-i1-GGUF)
Link to the original model:
[https://huggingface.co/Yuma42/KangalKhan-RawRuby-7B](https://huggingface.co/Yuma42/KangalKhan-RawRuby-7B)
To the people who prefer bigger models:
Im sorry but I only follow "small" large language models. And imatrix quants are very exciting for me in this regard.
What I like about KangalKhan-RawRuby: Seems to follow System prompt very well.
What I don't like: if you allow it to repeat itself to often, it won't be able to stop the repetition (If you know 7b models without repetition problems, let me know)
2 Comments
ab2377@reddit
Feztopia@reddit (OP)