Next Gemma versions wishlist

I had initially previously written this (or better, a variation of it), but it looks like the message became "hidden". Let me try again: - **Tone down the so-called "safety"**. Without extensive prompting, Gemma 3 is overly aggressive toward sending users to the su1c1de hotline (or similar hotlines) even for only very mildly inappropriate requests, which in my opinion has the opposite the effect of what it's supposed to do; it's also questionable whether such "suggestions" are even useful in the first place. Its hard avoidance of swearing and things like that even when instructed to is almost ridiculous for creative uses. - To be completely honest, most people here on LocalLlama most of all probably still want an open-weights alternative to character.ai that treats users like adults. You might have noticed that most model releases from the community deal with roleplay in a way or another. Gemma 3 Instruct on its own fares relatively nicely in this regard (after prompting), but there's room for improvement. Try to get Noam Shazeer (one of the Transformer authors, former character.ai CEO, now working at Google DeepMind) onboard and optimize the model for roleplay, informal conversations and similar uses. - Official support for a **system role**. That would remove much of the ambiguity and many quirks with prompting in its current form. Separating user-level from high-priority instructions would also be helpful in many cases where we want to limit or put constraints on how the model can react in response to user inputs. This should also be useful with "safety" where it's _actually_ needed. - Similarly, not just a general system role, but also official support for **system messages** placed at arbitrary points in the context would be helpful. For example, that could be used to drive/alter model behavior in real-time in more complex downstream applications. In practice this would be simply having the model support `system` role messages anywhere, and the `model` role to react to them with a higher priority compared to `user` messages. - **Hallucinations**. Gemma 3 hallucinates hard both in vision and text modalities when asked about something it is not 100% sure about. It's unclear if this is a result of its relatively good creativity; it would be a bummer if reducing hallucinations would make the model duller. - A larger, **more capable vision model**. The current one appears to have limitations, although this could also be the result of the implementation (e.g. in llama.cpp) not being up to spec with what the model can actually do. Is the Pan&Scan technique described in the paper actually working there, for example? Perhaps the llama.cpp maintainers need more help. - If increasing the vision model (or adding audio/video capabilities), try keeping total parameter size below 24B parameters or anyway a level where on current high-end GPUs the text model doesn't have to be crushed to low precision and the audio/vision can be retained in high precision, all while still allowing enough free memory for at least 32k tokens context. In this regard, Gemma-3-27B is probably still a tad heavier than it should optimally be on 24GB GPUs (RTX3090, RTX4090). - Try looking into ways for **reducing KV cache memory requirements**. Gemma 3 is a very "heavy" model compared to competing ones, which is strange considering that its various sliding window mechanisms are actually intended to make it save memory in this regard. This could be the chance for exploring the practical usage of alternative architectures as well (e.g. Google Titans).