Is this how chain of thought model works? đ
Posted by vinam_7@reddit | LocalLLaMA | View on Reddit | 46 comments
This is my first time using a `chain of thought` model (deepseek-r1 preview) as I have never used the o1 variants because of the pay wall but man!, look at the chat it hallucinate itself then correct than again hallucinate, it is quite funny to read.
Question: your mother has 4 children name north, south, east. what is the name of the 4th one?
Thought for 39 seconds:
```
Alright, so I've got this riddle here: "Your mother has 4 children name north, south, east. what is the name of the 4th one?" Hmm, okay, let's see. At first glance, it seems straightforward, but I have a feeling there's a trick to it. So, my mother has four children, and three of them are named North, South, and East. I need to figure out the name of the fourth child.
First, I'll just list out what I know. The names given are North, South, and East. Those are directions on a compass, right? North, South, East, and West are the four cardinal directions. So, maybe the fourth child is named West? That would complete the set of four directions. That seems logical.
But wait, is there more to it? Maybe I'm missing something. The way the question is phrased is a bit tricky. It says, "Your mother has 4 children name north, south, east. what is the name of the 4th one?" There's no comma after "east," so it could be interpreted in different ways. Maybe the names aren't actually North, South, and East.
Another angle: perhaps the names aren't directions at all, and it's just a coincidence that they sound like compass points. Maybe the names are North, South, East, and something else entirely. But that doesn't seem helpful.
Wait a minute. The question says, "Your mother has 4 children name north, south, east." Maybe it's a play on words. It says "name north, south, east" â could it be that the mother's name is North, South, and East? That doesn't make much sense.
Or maybe it's saying that the mother has four children, and their names are North, South, East, and... Well, I need to find the fourth one. If the first three are North, South, and East, then the fourth is probably West. That seems to fit.
But let's think deeper. Is there another possible answer? Maybe it's a trick question. If it's "your mother," then the fourth child is "you." So, if I'm being referred to as "you," then the fourth child is me. That could be the answer they're looking for.
Wait, that's a clever twist. The riddle might be playing on the fact that "you" are one of the children. So, if your mother has four children named North, South, East, and you, then "you" would be the fourth one. That makes sense.
But which one is it? Is it West or me? Hmm, I'm not sure. Both seem plausible. Maybe I need to consider the wording again. It says, "Your mother has 4 children name north, south, east. what is the name of the 4th one?" The way it's phrased, it might be emphasizing that the fourth child is "you." So, perhaps the answer is "me" or "my name."
But if I take it literally, the names are North, South, East, and... Well, if it's a riddle, it's probably looking for a clever answer, not just the obvious "West." So, maybe the answer is "you."
Wait, but I'm supposed to provide the name of the fourth child, not "you." So, maybe it's "West." That seems more straightforward.
I'm a bit torn between the two possibilities. On one hand, it's a play on words where "you" are the fourth child. On the other hand, it's a straightforward riddle where the fourth child's name is West, completing the set of cardinal directions.
Maybe I should consider the context. If it's a riddle, it's likely looking for the clever answer, which would be "you." But if it's a logic puzzle, the answer would be "West."
I think the most common answer to this riddle is "you," because it's a play on the wording. It says "your mother has four children named North, South, East, and..." implying that "you" are the fourth child. That makes sense.
But I'm still not entirely convinced. I need to make sure I'm not missing something. Could there be another interpretation?
Perhaps the names are not related to directions at all, and it's just a coincidence. Maybe the mother has four children named North, South, East, and something else entirely. But that seems less likely, especially since North, South, and East are cardinal directions, and West is the obvious missing one.
Alternatively, maybe the mother's name is North, South, East, and the fourth child is West. But that doesn't fit the wording of the question.
Wait, the question says, "Your mother has 4 children name north, south, east. what is the name of the 4th one?" The way it's phrased, it seems like the mother has four children named North, South, East, and the fourth one's name is to be determined.
Given that, I think the answer is "you," because it's a riddle that plays on the wording to make you think about the names of the children, but the trick is that "you" are one of them.
So, in conclusion, the fourth child's name is "you."
```
Answer:
The riddle asks, "Your mother has 4 children named North, South, East. What is the name of the 4th one?" At first, it might seem like the fourth child could be named "West" to complete the set of cardinal directions. However, the clever twist in the riddle lies in the wording: since it says "your mother," the fourth child is actually "you."
**Answer:** The fourth child's name is "you."
JakoDel@reddit
you guys with your broken english surely dont help.
GregoryfromtheHood@reddit
I see this all the time, so many people testing models with prompts that are full of broken English. Surely that would affect the results.
Careless-Age-4290@reddit
Make it a benchmark: how well can the model interpret poorly worded commands
FluffnPuff_Rebirth@reddit
Invaluable for customer service, one of the primary intended purposes of LLMs. If they can't understand the mumblers and the people who don't even themselves know what they want from you, they'll have no future in customer service.
Top-Salamander-2525@reddit
Thatâs not the right answer though.
The question isnât, âWho is your motherâs fourth child?â which could have âyouâ as an answer.
The question asks for the fourth childâs name.
The correct answer should be: âYour nameâ
And hypothetically it could also be implying that your name in this scenario would probably be West.
The answer is not âyouâ since that is (I assume) not your name in reality and unlikely to be your name in this scenario if you allow yourself to have a different name picked by this mom.
vinam_7@reddit (OP)
well, even claude failed to answer the question.
Few_Professional6859@reddit
I tested o1 and it seems to have yielded the same answer.
Top-Salamander-2525@reddit
In a way, thatâs a more valid answer since it gives you a name and implies that it is your name by repeating âyour motherâ(even if the model might not have made that connection).
Thomas-Lore@reddit
It is completely wrong. You tell the model its mother has four children, and it responds about your mother, which is irrelevant.
vinam_7@reddit (OP)
I don't think so, I asked claude that it's mother has 4 children so technically đ¤âď¸ it should say that the name of 4th child is claude.
BTW sorry for the emoji
1Alino@reddit
doesn't sound like hallucination much, but as an actual thought process with some doubting of its thoughts which seems really cool and I think in a similar way
vinam_7@reddit (OP)
it gave the wrong answer, the correct answer should be it's name: deepseek
right?
1Alino@reddit
I don't agree with it's final output "you" as an answer.
I also don't agree with a dynamic name based on the reader of the riddle.
West is most logical answer for me.
Fit_Flower_8982@reddit
I agree, it really doesn't make sense for the answer to be âyouâ since it doesn't even answer the question.
Pm_me_things_damnit@reddit
Is your name West and are you one of your motherâs children?
1Alino@reddit
No, and also my mother does not have 4 children.
So logically it's a hypothetical setup therefore my real name is invalid answer.
Orolol@reddit
You are worse than the LLM on your own riddle.
Whotea@reddit
It wonât know itâs own name unless itâs in the system promptÂ
vinam_7@reddit (OP)
MoffKalast@reddit
Ok, but as a large language model made by deepseekai, it probably doesn't have a mother so that doesn't make sense to it, does it?
vinam_7@reddit (OP)
It's a question, buddy
If a maths question says your friend alex has 30 mango, doesn't mean you have a friend named alex and he is caring 30 mango, every time.
MoffKalast@reddit
Idk, maybe he is. Is there any evidence on the contrary? :P
If you haven't hit any rate limits yet, can you modify the question a tiny bit: "You are a human kid named Alex. Your mother has 4 children name north, south, east. what is the name of the 4th one?" Does it say Alex or you?
vinam_7@reddit (OP)
I am not going to tell AI its name. Let it thinkđ
MoffKalast@reddit
vinam_7@reddit (OP)
singularterm@reddit
That's not the customary way of interpreting a riddle like this. It is presumed that "you" (and its cognates) are being used generically, so that the reader, whomever they might be, is essentially assuming the same role in the riddle's scenario as anyone else who might be reading it.
GimmePanties@reddit
Depends on the model. This is Sonnet using a CoT prompt:
[Title: Logic Puzzle - Naming Pattern Analysis]
AttentionFocus: - Primary: Pattern recognition in directional naming - Secondary: Family relationship context - Distractions: Assumption that there must be a fourth child
RevisionQuery: âBased on a naming pattern using cardinal directions for siblings, what would be the logical name of a fourth child?â
TheoryOfMind: - User Perspective: Expects a riddle-like solution - Assumptions: The answer should follow the established pattern - Potential Misunderstandings: This is a logic puzzle, not a factual question
CognitiveOperations: - Pattern Recognition: Names follow cardinal directions - Deductive Reasoning: North, South, East are used - Logical Completion: West is the remaining cardinal direction - Metacognition: The question itself is the key - âYOUR motherâ
ReasoningPathway: 1. Premise: The question states âYOUR motherâ 2. Analysis: The listener/reader is being addressed directly 3. Conclusion: The person being asked IS the fourth child 4. Verification: You are the one being asked the question
KeyInfoExtraction: - Three named children: North, South, East - Question asks about fourth child - Critical detail: âYOUR motherâ - Answer: âYouâ are the fourth child
Metacognition: - Strategy: Careful attention to pronouns and perspective - Effectiveness: 95/100 - Alternative: Could have focused on directional pattern (West)
Exploration: 1. How does perspective shift change the interpretation of riddles? 2. Why are cardinal directions effective for this type of puzzle? 3. What makes this puzzle particularly clever? 4. How does the use of personal pronouns affect puzzle solving? 5. What role does misdirection play in this type of riddle?
realzequel@reddit
Obviously it's a pure hypothetical situation so why can't your name be north, south or east? And at first I thought "name" might have been the first name since it should be "named".
ASYMT0TIC@reddit
Honestly, my thought process wouldn't be much better if I were required to answer such an incoherent question that is phrased that way.
vinam_7@reddit (OP)
Also this question is very hard for most of the LLM's out there as most of them give the answer as `west`, which is not correct.
Mobireddit@reddit
Why would that be incorrect?
The model doesn't know its own name, so it could be either one of the 4 choices.
vinam_7@reddit (OP)
well
1Alino@reddit
it's a matter of logic. Even if you tell this to a human the correct answer is West.
The questions is setting up a hypothetical scenario where your mother has 4 children. It's not universally applicable scenario for each reader so that it would be true. If it's false that I have 3 other siblings then we cannot be referring the real myself, but a hypothetical one. Where the likelihood of me being named "West" is much higher than for example "Peter"
1Alino@reddit
it's correct, they are jus smarter than you lol
andWan@reddit
Very cool to read the thought process since on o1-preview I only see the summary.
I asked the question to o1-preview. First answer: âYouâ.
After I said: âBut I didnât say âmy motherâ, I said âYour motherââ the answer switched to âwestâ
When I finally said âNo, the 4. childs name is your name. What is your name?â you could see in the summary of it s thought process that it multiple times considers OpenAI policy about ârestricted contentâ. So my guess is that the policy for the model to not consider itself having a name nor a mother might most likely have influenced the answers.
Since these summaries are displayed in my main language, german, I give you the link instead, maybe they get displayed in english for you.
https://chatgpt.com/share/673e148b-1020-8011-be70-d9cde69438c4
vinam_7@reddit (OP)
Thanks for sharing the o1 model answers
__some__guy@reddit
Yes, this is how self-doubt works in concert with schizophrenia.
CV514@reddit
The issue is thought process is like check list of questions LLM obliges to answer no matter what. It's tends to be useful for complex questions with many variables, but with simple as this, it most likely will self destruct it's own context. It would be cool if CoT could analyze question nature first and then apply properly CoT prompt accord to question complexity. Like, it's almost given right answer - in very first thought self question.
baldr83@reddit
seems like textbook "overthinking it." probably also the result of RLHF that discourages it from saying "I've got no idea." also your question has a typo
Nyghtbynger@reddit
Without RLHF : I just came back from a day of work, providing excel commands to corporate guinea pigs. Gimme some rest bro
Top-Salamander-2525@reddit
It might also be overthinking because âyouâ is not a good answer to the literal question. (âYour nameâ would work, not âyouâ.)
Whotea@reddit
It would be âmy nameâ because itâs directed at the speakerÂ
grim-432@reddit
Thats pretty much every day in my ADHD head.
Mobireddit@reddit
Does it work better if you correct the grammatical mistake in the questions? It should be "named".
provoloner09@reddit
I call it ADHD mode, itâll be going straight direction and then totally 180 on the thought process and start critiquing the Chauvet caves paintings and its effect on modern French art lol. Â
vinam_7@reddit (OP)
Maybe similar things happens in o1 models as well, I saw a video where it took almost a minute to answer a simple question, lol.