Stanford just dropped 5.5hrs worth of lectures on foundational LLM knowledge

This is showing a real lack of deep knowledge. Yes they both employ MLA, but Kimi K2 uses a new and different training algorithm. Specifically it uses the faster and more efficient MuonClip training algorithm. It also has fewer dense layers and attention heads. It's larger but with less active parameters. LongCat has Shortcut connected Mixture of Experts with a variable number of active parameters so that it dedicates the most compute power to the hardest to generate tokens. They also clearly train them on different tasks and data as Kimi is a significantly better coding model than DeepSeek. Training pipeline is just as important as architecture to making a good model. GLM 4.6 took the world by storm despite being on the smaller side and architecturally quite boring. Only interesting thing it did architecture wise is employ multi-token prediction, but that's something DeepSeek can also do. Otherwise it uses a fairly archaic GQA based mechanism. The reason it's so good is the training. My point anyway was not how novel they are but the fact that DeepSeek is not the only MLA model.