Gemma 4 base GGUF?
Posted by Sh0w_T1mer@reddit | LocalLLaMA | View on Reddit | 9 comments
Hello, I've seen reviews that gemma 4 31b base is very good at roleplaying. But I can't find the gguf version of the basic gemma 4 anywhere. Only the instructions version is available everywhere. Where can I find a quantized gemma 4 base?
uti24@reddit
Don't you need instruction tuned version for RP anyways?
Herr_Drosselmeyer@reddit
Not necessarily. It depends on the model, but base versions definitely can understand instructions too.
uti24@reddit
I understand that, my questions is why they want specifically base version for RP
Herr_Drosselmeyer@reddit
Maybe they think that base versions can be more creative. But that's so subjective that it's hard to tell. Most people generally prefer instruct tuned models though.
Top-Rub-4670@reddit
If you go to https://huggingface.co/google/gemma-4-31B
Look on the left column, you'll find a link to https://huggingface.co/models?other=base_model:quantized:google/gemma-4-31B
I guess you looked everywhere except the obvious place 🤷♀️
bharattrader@reddit
You can try https://huggingface.co/mradermacher/gemma-4-31B-GGUF
aeqri@reddit
There's plenty at https://huggingface.co/models?other=base_model%3Aquantized%3Agoogle%2Fgemma-4-31B&sort=downloads
bharattrader@reddit
https://huggingface.co/mradermacher/gemma-4-31B-GGUF
FusionCow@reddit
I made a q3_k_m with llama.cpp, https://huggingface.co/FusionCow/gemma-4-31B-Q3_K_M-GGUF, but you might want a higher quant or something