Who is Elara?
Posted by itsjustmarky@reddit | LocalLLaMA | View on Reddit | 41 comments
I run GLM 4.5 Air FP8 locally, and I have this really bizarre artifact that keeps happening that I have yet to figure out.
One of the tests I like to do with models is just ask it to "tell me a short story'. I use this typically to get a rough idea of how many tokens/second I get and to just make sure everything is working.
When I do this test with GLM Air 4.5 FP8, the official zai-org/GLM-4.5-Air-FP8 no less, I constantly see "Elara" as a character name very frequently. Like it can be 4-5 times out of ten stories at a time. At first I thought it might be an artifact of Cherry Studio maybe injecting something into the system prompt, but I see it in LibreChat as well.
For example, I just asked for a short story in Cherry Studio and LibreChat, and I got the name in both back to back. It comes up so often, that I ran the same prompt 10 times in cleared context and noticed the name at least 4 different times.
As far as I have noticed, it's the only thing that seems to crop up frequently but I haven't done extensive research on this. I have however run through a lot of testing, and get very high 97% reasoning scores over hundreds of tests. So I know things are working.
My setup is SGLang, running GLM 4.5 Air FP8 with included template.
This is the only strange thing I have noticed, if I ask it to code or do any legit work, it all seems normal. If I ask for short stories, it seems to heavily favor the name Elara and I don't have a clue why.
This is a short story from Anything LLM, but the first story had no name, the second I saw Elara pop up again. Three different front ends, same artifact.
MixtureOfAmateurs@reddit
Real ones remember Sarah
-Ellary-@reddit
Everyone know Elara, don't be silly.
But we need to talk about Lord Gray.
KitchenFalcon4667@reddit
Use Olmo as it has trace to where in training data things comes from: https://playground.allenai.org/thread/msg_P7M9J6G3I2?selectedMessage=msg_G8H8N2Z0B5
IngenuityNo1411@reddit
I really did some google searches to find out its root before ChatGPT released (2020), then it feels even more strange:
here https://www.reddit.com/r/litrpg/comments/i06lvh/eloria/ which says
And this, a work MAYBE written before ChatGPT: https://www.wattpad.com/story/182833430-the-tail-of-the-sword-and-shield-wattys2020
So... something went wrong even before we use LLMs?
FastDecode1@reddit
I don't think this is very complicated.
If you do a Google Books search for anything released up to the year 2019, you get about a million results (estimate given at the bottom of the Tools menu).
And if you search Wikipedia to find out if Elara is a real name, you find out that it's literally the name of a god from Greek mythology as well as an ancient Indian king.
Anyone who has consumed speculative fiction of any kind knows that writers are some of the least creative people when it comes to naming things (second only to people who actually name things IRL). They mostly just recycle names from mythology/folklore, history, and other literature. If you draw a Venn diagram of mythology, history, and literature, and at the intersection you'll find Elara as well as 99% of all other "fantasy" names.
As for why LLMs (allegedly/seemingly) use this name a lot, it's probably due to recency bias. Elara is both an ancient deity and a king, and hasn't been used in recent mainstream literature. A writer would think it's an absolute gem of a name and would sprain his arm patting himself in the back for coming up with it.
Feztopia@reddit
Why do you expect a computer program to do something different when it's asked the same thing twice? Yes there is some rng going on with sampling and rounding at the hardware level. But if the model decided during it's learning that name x has the highest probability for a story, why should that change if you ask it multiple times?
Koksny@reddit
Well to be perfectly honest they are probably using different seed each time. In a perfect world, with different seed and non-greedy decoding, it shouldn't happen.
Feztopia@reddit
A perfect model could still have a preference just not that high, because some names in the real world are also much more popular than others. I'm sure you know it, but just to say it, changing the seed changes the whole output not the probability of the single tokens. Other tokens than Elara are more likely to change which makes Elara stand out.
itsjustmarky@reddit (OP)
by your theory I should get the exact same output every time
Feztopia@reddit
I already gave you the sources of the rng, if you can disable these you will always get the same answer. For some things the model learns to have a stronger opinion than for others. Disable the rng in hardware and software (not easy) and you always get the same most probable result.
spaceman_@reddit
No, we deliberately add randomness to the system to avoid that, but the probabilities are the same every time and it seems the probability for Elara is pretty high.
toothpastespiders@reddit
A probable token matcher matching probable tokens isn't surprising. 'That' being the probable token combination over anything else is the weird part.
Feztopia@reddit
Yes but reading the post it sounds to me more like OP was surprised that he got the same name not that specific name. Like he was first thinking the software was injecting the name, you wouldn't think that if you knew that that's not surprising behavior.
-p-e-w-@reddit
This phenomenon has been observed for at least three years and despite handwaving “explanations”, nobody really knows what’s going on.
Elara is not a common character name in fiction, just like Eldoria is not a common place name, and yet those two names keep popping up in LLM-generated stories. We don’t know why, and saying “it’s because they are the most probable names” is just restating the question.
Spirited_Bag_332@reddit
The combination of those letters is quite common so I think it has a high probability to end in this or similar names. I learned this when I was developing my personal (non-AI) name generator.
Koksny@reddit
It's extremely popular name in all kind of fanfics. When amateur writers try to be creative, they are very likely to hit cliche, and it's one of them.
-p-e-w-@reddit
Since a huge percentage of fanfics are set in the Harry Potter universe, you would expect models to constantly use the names Hermione and Ginny. But they don’t.
Just an example showing why such simplistic explanations fall flat.
It’s okay to not understand something, and this is clearly something that we currently don’t understand.
Koksny@reddit
I think You are underestimating the number of fanfics in general, compared to any singular universe. And if You add something like "Hogwarts" somewhere in prompt, they definitely do.
When You ask someone for a random number, they will probably give you 7, or 23, or something that 'sounds random'. If You ask me for a random uncommon female name, i would probably say something like 'Elara', simply because i don't know any.
Most like Meta used dataset over-saturated with the name for training Llama3, and all the other models snowballed from it. If there is any mystery in it - maybe they've tried to sanitize the dataset, and overdid it with this particular name. It's like asking why all the models use the same 'shivering in the spine', 'it's not just X, it's Y', and so on - it was just trained on cliche.
itsjustmarky@reddit (OP)
I would think Bob and Christopher would be way more common than a single niche as in fanfic. Yet you never see those names either.
Koksny@reddit
The problem is, without actual data that proves some statistical anomaly, this is basically just anecdotal. And i bet that If You see name like "Elara" or "Lillith", You are just much more likely to notice it compared to "Alice" or "Kate".
All i can say, we've worked on project that generated random NPCs, including their names - and we had to resort to an array and just generating random number to select from it, because models like Gemma or Llama would always give us the same 3 to 5 names, no matter the seed.
phayke2@reddit
You would think if a made-up name like that is so popular within writing circles online, even just by coincidence, that it would be a more popular baby name in the first place.
AppearanceHeavy6724@reddit
This is a very bad "handwavy", "smart-alec" explanation, which is also wrong. Elara is a only a tip of iceberg; there are many cliches present in the outputs of the models that have frequency far above than in normal text, and the data used for training does not have these biases. There is something strange going when LLMs latch to certain linguistic constructs and overuse them while maintaining reasonably fluent, normally looking text overall.
Koksny@reddit
And You base this statement on what exactly? How many models are there that have open source datasets?
Phi is trained on synthetic only data.
AppearanceHeavy6724@reddit
I base that on the recent research by the team who made Gemma 3 antislop models. AFAIr they used massive amount of data from reddit /r/writingprompts as the baseline fir human writing. Most of the linguistic constructs followed the frequency distribution in human writing, yet some were massively overrepresented.
Let me know what fanfic dataset contains smell of ozone every time something happens.
Have actually read what I wrote, fam? Again: these two models are same weights ran once and twice respectively, yet they have different slop patterns. Do you understand what that means, or you want me to eli5?
Pan000@reddit
I believe this occurred because OpenAI changed the names of the characters in books when using them for training data. This was then used to train GPT 3.5 Turbo, which was in turn used to generate synthetic training data by everyone.
The infection is now rooted. It exists literally in the base models, all of them I have tested. It's actually worse in the base models than instruct tuned models.
Generic_Name_Here@reddit
https://www.reddit.com/r/MaliciousCompliance/s/LRgaQW16lP
“If your character’s name is Elara, -99 points”
phree_radical@reddit
The model is the dataset
https://www.google.com/search?q=site:huggingface.co+%22elara%22
AppearanceHeavy6724@reddit
Those datasets are partialky synthetic, yet Mistral had claimed that Mistral Small 3 had no synthetic data when they released it. Guess what - it was sloppiness most Elarious model I've seen.
phayke2@reddit
That's Elarious 😂
AppearanceHeavy6724@reddit
If you ve never checked OG Mistral Small 3/3.1 (not 3.2) it is an absolute king of slop. The worst caricature of AI creative writing.
Torodaddy@reddit
Its an artifact from these models using the same training corpus, why that name? Probably as people have said it read a lot of edgelord fanfic
fluffywuffie90210@reddit
This smells like ozone.
Do any fantasy-type storys and you'll see the same names pop up time and again. Both amusing and annoying, it's hard to create memorable characters!
Klutzy-Snow8016@reddit
Imagine if I put you in a cryogenic pod, and every time someone wanted to talk to you, I made a new temporary clone and shoved it out the door. Each new clone has no knowledge of anyone's conversations with previous ones. If someone asks you to write a story, you have to choose character names. So you pick names that you would be likely to pick, for whatever reason. Probably something that sounds like it fits based on the vibe of the story and your own idiosyncrasies. Even if you were trying to be random, you wouldn't actually be, because everyone has built-in biases.
I imagine it's similar for LLMs. At least, the part where they start cold every time and don't know they've said "Elara", "Dr. Aris Thorne", or "Silas Vance" in 100 previous conversations already. They're built to analyze the prior text and produce, for each token in the vocabulary, a probability that that token is the next one that should go there. When it comes to a spot for a character name, it has to pick something. A name could technically be anything. "Cool-Looking Glyph: the Artist Formerly Known As [Elon Musk's son's name]" is a valid name, but that's unlikely. Some possible names are going to be more likely than others.
Why do LLMs come up with the particular names that they do? I don't know. But it at least feels to me like it makes sense that an LLM would have a small set of go-to names that "seem like" they fit. Disclaimer: I don't actually have any real insight.
eloquentemu@reddit
I suspect that during SFT they use synthetic data for responding to "write a story" which sort of makes sense since they want to establish good grammar, consistent style, etc. Somewhere along the line, Elara randomly came up and got reinforced as more synthetic data and RL steps happened.
For GLM 4.6 I found I can drop something like:
In the system prompt to add some entropy to name generation. It'll sometimes drop an Elara but it helps a lot.
Koksny@reddit
There is limited number of names. On clean prompt, you are bound to hit the most likely 'random' one.
You can fingerprint models (or even particular version of a model) by asking them for random number in given range, they are quite likely to always favor one or two. That's basically the same thing.
itsjustmarky@reddit (OP)
Limited number of names, like only 4?
roosterfareye@reddit
42
mineyevfan@reddit
More than five, even
Koksny@reddit
From my own experience with models like Llama - yeah, unless prompted otherwise, You will see the same small roster over and over again, with some variation depending on 'age' of the character.
Dr_Lipschitzzz@reddit
Qwen3-30b-A3B also uses Elara alot when naming female characters. So weird that multiple llms do the same
itsjustmarky@reddit (OP)
that's bizarre