What word ends in three e?

Posted by Barafu@reddit | LocalLLaMA | View on Reddit | 18 comments

I found a question to befuddle all the LLMs I could try it on. "What dictionary word ends in three е?" First, try answering it yourself. Every kid I know can answer it. In fact, if you are a kid, it feels like every adult is obligated by law to ask you this. Second, ask an LLM. But make sure you type it, don't copy-paste it. See them get confused. I don't have access to the top price models, but everything else offers "Bree" or "wee" or something like that. Now, in a new chat, ask again, but copy-paste the question from here. Get the answer immediately.

Reply to Post

18 Comments

[-]

CluelessOuphe@reddit

What adults are asking this question? Is this entire post a hallucination? I can't think of an answer myself.

[-]

TastesLikeOwlbear@reddit

I'm with you. This post is nonsense.

[-]

BobbyL2k@reddit

Spelling questions are difficult for LLMs since they don’t see the characters we type to them. They see each word as an abstract concept that is inferred by their usage. To fix this gap, the model providers will have to train this knowledge in specifically, which will definitely not generalize due to the architectural reasons I stated above. So you will always find gaps in any LLM related to spelling. This test doesn’t tell us anything besides the fact that LLMs can’t see how a word is spelled, which every researcher already knows.

[-]

PeakBrave8235@reddit

It tells us that transformer models are pattern matching algorithms, text prediction. People claiming intelligence are artificially intelligent themselves lmfao

[-]

Pristine-Woodpecker@reddit

>claiming intelligence What's your definition of intelligence?

[-]

PeakBrave8235@reddit

Apparently yours is token prediction holy shit. Go talk to your keyboard I guess

[-]

BobbyL2k@reddit

I agree about the pattern matching part. But it has nothing to do with intelligence. A human can’t see infrared but can sort of feel the heat. So an alien asks us, which item in from of them is hot and the human answers incorrectly. Is that human unintelligent? Maybe? But that has nothing to do with the answer since humans can’t biologically see infrared. Same for LLMs, they can’t see the characters. You could argue humans can develop tools that can see infrared. But so can LLMs write simple snippets of code that count or inspect the characters in a word.

[-]

Pristine-Woodpecker@reddit

>So you will always find gaps in any LLM related to spelling. French grammar for example has a few rules about words starting with vowels. Ask an LLM, even SOTA, to explain a few cases with examples, and you get nonsensical answers because the LLM can't "see" the individual vowels. But they can translate and speak French without issue because they know what words go together.

[-]

PeakBrave8235@reddit

Exactly

[-]

Qazax1337@reddit

How are people still amazed LLMs are not good at spelling.

[-]

Barafu@reddit (OP)

I am demonstrating that they are also not good at stopping specific words and topics.

[-]

Qazax1337@reddit

My original question still stands

[-]

Apprehensive_Plan528@reddit

A paid model gives me this: In standard English spelling, there are essentially no ordinary dictionary words that end with the exact three-letter sequence “eee”. What you do see are things like: • Onomatopoeic or playful spellings people improvise in text: “yeee,” “wheee,” “reee.” • Usernames, brand names, or codes constructed with “eee.” But in normal, recognized English vocabulary lists, endings are -e, -ee, -ie, etc., not -eee.

[-]

korino11@reddit

In russian dictionary it will be - длинношеее

[-]

Green_Burn@reddit

Conduit and Shvambrania is a great book

[-]

lemondrops9@reddit

I copied and pasted you question and it went nuts. I haven't tried a 2nd time yet.

[-]

Barafu@reddit (OP)

Deepseek by API, gpt-oss-120b and some 70b models in LMStudio.

[-]

x11iyu@reddit

first of all, everyone knows for quite a while now that due to the way LLMs tokenize words, they don't work well on character-level tasks. [BLT](https://arxiv.org/pdf/2412.09871) was made to address this, tho I think I read somewhere that the authors themselves abandoned the idea as it doesn't scale well? second, apparently I'm an LLM as well, because I can't for the life of me figure out what word ends in "eee." unless the answer you're looking for is, there is no such word.