(Quick links in case you don't know the meme or what LARP is)

If you only ever read by top/hot and not sort by new then you probably don't know what this is about, as postings with that content never make it to the top. Well, almost never.

Some might remember the Qwen3-Coder-30B-A3B-Instruct-480B-Distill-V2 that made it to the top two months ago, when many claimed that it was a great improvement. Only after extensive investigation it was proven that the new model wasn't (and could have never been) better. The guy who vibe-coded the creation pipeline simply didn't know what he was doing and thus made grave mistakes, probably reinforced by the LLM telling him that everything is great. He was convinced of it and replying in that way.

This is where the danger lurks, even though this specific case was still harmless. As LLMs get better and better, people who lack the domain-specific knowledge will come up with apparent great new things. Yet these great new things are either not great at all, or will contain severe deficiencies. It'll take more effort to disprove them, so some might remain unchallenged. At some point, someone who doesn't know better will see and start using these things - at some point even for productive purposes, and that's where it'll bite him, and the users, as the code will not just contain some common oversight, but something that never worked properly to begin with - it just appeared to work properly.

AI slop / psychosis posts are still somewhat easy to identify. Some people then started posting their quantum-harmonic wave LLM persona drift enhancement to GitHub, which was just a bunch of LLM-generated markdown files - also still easy. (Btw: Read the comments in the linked posts, some people are trying to help - in vain. Others just reply "Stop LARPing" these days, which the recipient doesn't understand.)

Yet LLMs keep getting better. Now we've reached the stage where there's a fancy website for things, with code on GitHub. Yet the author still didn't understand at first why their published benchmark isn't proving anything useful. (Btw: I didn't check if the code was vibe-coded here, it was in other - more extreme - cases that I've checked in the past. This was just the most recent post with code that I saw)

The thing is, this can apparently happen to ordinary people. The New York Times published an article with an in-depth analysis of how it happens, and also what happened on the operations side. It's basically due to LLMs tuned for sycophancy and their "normal" failure to recognize that something isn't as good as it sounds.

Let's take DragonMemory as another example, which caught some upwind. The author contacted me (seemed like a really nice person btw) and I suggested adding a standard RAG benchmark - so that he might recognize on his own that his creation isn't doing anything good. He then published benchmark results, apparently completely unaware that a score of "1.000" for his creation and the baseline isn't really a good sign. The reason for that result is that the benchmark consists of 6 questions and 3 documents - absolutely unsuitable to prove anything aside from things being not totally broken, if executed properly. So, that's what happens when LLMs enable users to easily do working code now, and also reinforce them that they're on to something.

That's the thing: I've pushed the DragonMemory project and documentation through the latest SOTA models, GPT 5.1 with high reasoning for example. They didn't point out the "MultiPhaseResonantPointer with harmonic injection for positional resonance in the embeddings" (which might not even be a sinusoid, just a decaying scalar) and such. The LLM also actively states that the MemoryV3Model would be used to do some good, despite being completely unused, and even if it would be used, then simply RoPE-extending that poor Phi-1.5 model by 16x would probably break it. So, you can apparently reach a state where the code and documentation look convincing enough, that a LLM can no longer properly critique it. If that's the only source of feedback then people can get lost in it.

So, where do we go from here? It looks like things will get worse, as LLMs become more capable, yet still not capable enough to tell the user that they're stuck in something that might look good, but is not good. Meanwhile LLMs keep getting tuned for user approval, as that's what keeps the users, rather than telling them something they don't want or like to hear. In consequence, it's becoming more difficult to challenge the LLM output. It's more convincingly wrong.

Any way out? Any potentially useful idea how to deal with it?