Do you think Gemini 3 uses MoR or Titans?
Posted by SrijSriv211@reddit | LocalLLaMA | View on Reddit | 4 comments
Google some time ago introduced MoR and Titans
I'm very curious if Gemini 3 uses a combination of these two architectures.
I know that Google has described Gemini 3 as a sparse MoE transformers-based model but Google also described MoR as a transformers-based architecture which using recursion and dynamic token routing. So what if Gemini 3 is a mixture of MoR, Sparse MoE, Titans and Transformers, or am I missing something?
What are your thoughts on this?
Medium-Ad-9401@reddit
I think that they have already implemented some of these ideas, if not, then the result is even more monstrous, because this means that the next version, which will be based on all these ideas, will simply destroy everything.
SrijSriv211@reddit (OP)
Yeah. Demis in one podcast said that sometimes some ideas can't be implemented into next iter of the model so they keep it for the next of next iter.. Gemini 3 is this good, I wonder how amazing 3.5 and 4 will be. I was honestly so happy & surprised because the launch was totally out of the blue for me.
ELPascalito@reddit
I mean, they released the model card, it only specifies that it's a sparse mixture-of-experts, so I presume no
SrijSriv211@reddit (OP)
Sad. It'd have been cool to see more experiments or models on Titans or MoR from Google..