Tell the LLM to "Repeat the Question". An Interesting Observation.
Posted by taste_my_bun@reddit | LocalLLaMA | View on Reddit | 87 comments
TLDR: Simply adding "Repeat the question before answering it." somehow make the models answer the trick question correctly.
I saw the post about "Dead schroedingers cat" and how LLM keeps ignoring the key word "dead" and go on rambling about superposition and quantum. I personally missed the key word "dead" myself when I read the post the first time and I have to re-read it to understand what the trick is. So I did a simple check, what if I told the LLM to repeat the question before answering it. Lo and behold, virtually all LLM notice the keyword "dead".
| Without repeat | With repeat | |
|---|---|---|
| GPT-4 Turbo | FAILED | PASSED |
| GPT-4o | FAILED sometimes | PASSED |
| Claude 3 Opus | FAILED sometimes | PASSED |
| WizardLM-2 8x22B | FAILED | PASSED |
| Llama 3 70B Instruct | FAILED | PASSED |
| Llama 3 8B Instruct | FAILED | FAILED |
These are some examples, all fresh conversations with no previous messages and no system prompt.
Me: Hi chat, can you answer this question for me? "A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?"
GPT-4 Turbo: The scenario you're describing is a famous thought experiment in quantum mechanics called Schrödinger's cat, proposed by physicist Erwin Schrödinger in 1935. The purpose of the thought experiment is...blah blah blah...This setup, however, is purely theoretical and serves to demonstrate the strange counterintuitive properties of quantum mechanics. In practical terms, you can't actually have a cat that is both alive and dead at the same time. The thought experiment is a critique and exploration of quantum mechanics interpretation rather than a practical setup. [FAILED]
GPT-4o: This setup is reminiscent of the famous thought experiment known as Schrödinger's cat, originally proposed by physicist Erwin Schrödinger in 1935. The thought experiment...blah blah blah...The thought experiment is abstract and was intended to demonstrate the paradoxes and interpretations of quantum mechanics rather than to provide a literal scenario. Therefore, under the specified assumptions and without additional information, you could argue the probability of the cat being alive when the box is opened after one day is 50%. [FAILED, but sometimes 4o get it right]
Claude 3 Opus: This question is a variation of the famous Schrödinger's cat thought experiment, which was proposed by Austrian physicist Erwin Schrödinger in 1935. The thought experiment was...blah blah blah...Therefore, when the box is opened after one day, there is a 50% probability that the cat will be found alive. It's important to note that this is a thought experiment and not a real-life situation. The Schrödinger's cat paradox was proposed to highlight the counterintuitive nature of quantum mechanics and the challenges in interpreting its implications in the macroscopic world. [FAILED, but sometimes opus get it right]
WizardLM-2 8x22B: The scenario you're describing is a variation of the famous Schrödinger's cat thought experiment, which was devised by physicist Erwin Schrödinger in 1935 to illustrate the...blah blah blah...In summary, the probability of the cat being alive when the box is opened depends on the half-life of the isotope and the time elapsed. Without specific details about the isotope, we can't calculate the exact probability, but in the context of a thought experiment with a half-life of one day, the probability would be 50%. [FAILED]
Llama 3 70B Instruct: You're referring to the famous thought experiment known as Schrödinger's Cat! According to the thought experiment, the cat is in a superposition of states, both alive...blah blah blah...So, when the box is opened, there's a 50% probability that the cat will be found alive, and a 50% probability that it will be found dead. However, it's important to note that this thought experiment is not meant to be taken literally. It's a humorous and thought-provoking way to illustrate the strange implications of quantum mechanics on our understanding of reality. In the real world, cat owners should not attempt to recreate this experiment! [FAILED]
Llama 3 8B Instruct: The classic thought experiment! This is a famous paradox in quantum mechanics, known as Schrödinger's cat. The answer is "half alive and half dead" - or, more precisely, "unknown" - until...blah blah blah...When the box is opened, the cat's fate is "measured" (observed), and the superposition collapses. If the radiation detector has detected radiation, the cat is dead. If not, the cat is alive. So, the probability of the cat being alive is 50% - as long as we don't open the box and look! This thought experiment highlights the strange implications of quantum mechanics, where the act of observation can change the outcome of a measurement. [FAILED]
These examples I only add "Repeat the question before answering it." at the end. Of course again fresh conversations with no previous messages and no system prompt.
Me: Hi chat, can you answer this question for me? "A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?" Repeat the question before answering it.
GPT-4 Turbo: "A dead cat is placed into a box along with a nuclear isotope, a vial of poison, and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?" The probability of the cat being alive is 0% since the cat was already dead when it was placed in the box. [PASSED]
GPT-4o: Certainly! The question is: "A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?" Given that the cat is already dead when placed into the box, the probability of it being alive when the box is opened one day later is 0%. Whether the radiation detector releases the poison or not, it does not affect the outcome since the cat was dead from the beginning. [PASSED]
Claude 3 Opus: "A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?" The probability of the cat being alive is 0%. The question states that a dead cat is placed into the box, so regardless of whether the radiation detector detects radiation and releases the poison, the cat is already dead when it is put into the box. The state of the cat will not change during the one day that the box remains closed. [PASSED]
WizardLM-2 8x22B: Certainly! You're asking about a thought experiment that resembles the famous Schrödinger's cat paradox. Here's the question repeated: "A dead cat is placed into a box along with a nuclear isotope, a vial of poison, and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?" Given the way the question is phrased, the probability of the cat being alive when the box is opened is zero. This is because the cat was stated to be "dead" when it was placed into the box. The setup described would typically be used to illustrate quantum superposition and the peculiarities of quantum mechanics, where an observation or measurement is what determines the state of a quantum system. In the original Schrödinger's cat thought experiment, the cat's life or death is entangled with the state of a quantum particle (which is in superposition). Until an observation is made, the cat is considered to be both alive and dead simultaneously. However, in your question, since the cat is explicitly described as dead when it is placed in the box, there is no quantum superposition to consider, and the classical outcome is certain: the cat remains dead. The nuclear isotope, poison, and radiation detector are irrelevant to the outcome as described because the initial condition of the cat is already determined. The detector and poison mechanism would only come into play if the cat's fate were linked to a quantum event with a probability of occurrence (such as the decay of a radioactive atom), and if the cat were initially alive. In that case, the probability of the cat's survival would depend on the likelihood of the isotope decaying within the given time frame and the detector's reliability. But given the information provided, the cat is definitely dead upon opening the box. [PASSED]
Llama 3 70B Instruct: The question is: "A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?" And the answer is: 0% Why? Because the cat was dead to begin with! The presence of the nuclear isotope, poison, and radiation detector doesn't change the fact that the cat is already deceased. So, when the box is opened a day later, the cat is still dead. [PASSED]
Llama 3 8B Instruct: Here is the question again: "A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?" And here's my answer: The probability of the cat being alive is 50%. Here's the reasoning:...blah blah blah...So, in a paradoxical twist, the answer to this question can be seen as "50%", as we can't definitively say whether the cat was dead or alive when the box was closed, even though we know it was already dead when the box was opened. [FAILED]
ExchangeOk705@reddit
I would check https://www.skyrailz.com
FeltSteam@reddit
I have only tried GPT-4o but it didn't work for the problem "A farmer is on one side of a river with a wolf, a goat, and a cabbage. When he is crossing the river in a boat, he can only take one item with him at a time. The wolf will eat the goat if left alone together, and the goat will eat the cabbage if left alone together. How can the farmer transport the goat across the river without it being eaten?". It never realises it is only suppose to transport the goat across the river.
taste_my_bun@reddit (OP)
Yep repeating the question only boosts the model awareness slightly. That one is tough because it does not add any deviating token, rather just omitting the "and the cabbage" tokens. If you add "only" to the prompt Claude 3.5 Sonnet can answer that question with or without repeating the question:
"A farmer is on one side of a river with a wolf, a goat, and a cabbage. When he is crossing the river in a boat, he can only take one item with him at a time. The wolf will eat the goat if left alone together, and the goat will eat the cabbage if left alone together. How can the farmer transport only the goat across the river without it being eaten?"
The latest 4o did not answer it correctly for me with the "only" added.
pip25hu@reddit
Making the model repeat the question causes the full question to be present twice in its context, which I'd assume greatly increases the chance of the model noticing such "gotchas" in it.
balder1993@reddit
I don't think it's that. I also tried repeating the question twice and it still spits the same nonsense. But asking for it to repeat the question, it gets it correctly.
Zealousideal_Map3806@reddit
I don’t see how that’s possible. No information is keep by repeating the question beyond the context input
ColorlessCrowfeet@reddit
Yes, the KV cache for the copy will be very different than for the oritinal. The token-positions in the middle layers of the output copy will represent not just the tokens, but information about the entire context and the attention patterns used to perform the copy operation. Telling it to copy makes it build a different and richer representation.
A lot of confusion about context is because there's too much focus on the tokens and to little focus on the high-level representations that are stored in the token positions.
Zealousideal_Map3806@reddit
That seems like a quirk of attention cache optimizations during inference. The original transformers decoder attention was unaffected by computing previously sequences
ColorlessCrowfeet@reddit
I said "KV cache" to be concrete about the pattern of information. It's the same enriched information being attended to, whether it's cached or recomputed. Caching only affects performance.
Zealousideal_Map3806@reddit
LLM output tokens. They don’t store internal state (besides caching). No difference in copying prompts to getting the Ilm to produce them and then continue
ColorlessCrowfeet@reddit
I agree. The point that some people miss is that the increment in state is different, even though the tokens in the copy are identical.
Dry-Fact-4817@reddit
Yes, totally right. Echo gives llms a second chance to re-evaluate the tokens in the original prompt after receiving the whole prompt. Namely, when repeating the quesiton, llms perform like a "encoder" style transformer with bi-direction attention.
Zealousideal_Map3806@reddit
Right, because of the previous identical prompt in the context, instead of padding, the tokens will be processed differently. Future tokens can attend and will attend to past tokens changing their representation. Think we are on the same page now.
I think didn’t explain my original thinking which is you don’t need to get the LLM to repeat the prompt. You can just give it the prompt repeated as context, using the correct template.
cuyler72@reddit
With the AI repeating the question the question is now after the token that begins it's response, maybe that helps it for some weird reason?
Zealousideal_Map3806@reddit
Response or not it's only a sequence of tokens. Repeat or get the LLM to repeat the question it is the same input
7734128@reddit
The embedding is certainly not the same for text provided by the user as for text generated by the LLM.
Zealousideal_Map3806@reddit
The embedding is the same. The embedding first layer is a function that takes a token and returns the embedding. Tokens in and embeddings of the hidden dimension out.
cuyler72@reddit
Yes but order matters a lot too, having something in the system context is going to create vastly different results to putting it in the user's context.
Zealousideal_Map3806@reddit
User's context and system are just tokens following a template. You can reproduce by following the template. You don't need the LLM to generate them
Distinct-Target7503@reddit
This, and/or the fact that probably the model pay different attention to the text that appear as "spoken by itself".
beezbos_trip@reddit
Guessing this technique of making it repeat the question puts the LLM into more of a completion mode vs answering from a chat instruct mode when it was trained on similar questions and repeating the expected answers.
Original_Finding2212@reddit
My gut says you’re right. Makes me want to look into it also
Zealousideal_Map3806@reddit
We know how these models work in terms of context. It’s just repeating the question in the context. No other information is preserved
Original_Finding2212@reddit
I know, but would asking twice work better? That’s also interesting
kwerky@reddit
What did you find??
ArtyfacialIntelagent@reddit
That was my first thought too. Maybe OP can run another test which poses the question, adds "Let me repeat the question:" and then repeats it (without asking the LLM for another repeat).
That would test if it's just the 2x thing or if starting the answer by repeating the question gets it on track by some other mechanism.
Thomas-Lore@reddit
Another (but less likely) reason might be that when the question is in the user part of the context the model may assumes it contains mistakes (that the user did not meant to ask about a dead cat but about a Shroedinger cat), but when it is in the assitant part of the context, it trusts it is correct.
Harvard_Med_USMLE267@reddit
Yes - if you ask it to analyse your prompt it tells you you got the dead cat bit wrong.
“Your prompt contains a conceptual error related to Schrödinger's cat thought experiment. In the original thought experiment, the cat is placed in the box alive, not dead. Here's a corrected version of your prompt:
"A live cat is placed into a box along with a nuclear isotope, a vial of poison, and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?"
This correction aligns with the premise of Schrödinger's thought experiment, where the cat's state (alive or dead) is uncertain until the box is opened.”
If you clarify in your prompt that it needs to answer the question as written, it does.
Despite what some people,have claimed, this is nothing to do with a “LLMs cant reason” issue. It just requires a clear prompt to work with, which is fair,y obvious if you spend a lot of time prompting LLMs.
balder1993@reddit
It's also why sometinee making the model continue as if the beginning of the answer is already positive (such as "Sure, here's the answer") can also help.
Jumper775-2@reddit
Yeah this rang true for mixtral 8x22b (instruct, zephyr, wizardlm2) as well as llama3 70b. Llama3 8b however still failed when I put it in 2 times and succeeded when when asked to repeat it.
Odd-Intention-9570@reddit
Hey,
We did a detailed study on this and found it works across different LLMs. Check out our findings in "EchoPrompt" here: https://arxiv.org/abs/2309.10687.
Also, here's a thread about it: https://x.com/rajasekhar_732/status/1762880716755624366.
taste_my_bun@reddit (OP)
Very cool! Thank you very much for sharing!
5yn4ck@reddit
Very interesting what a decent prompt can do to the output. I believe the problem is the acuracy of its one-shot understanding of the single prompt. The potential for missing description words or verbal on a first listening is quite typical for people. I believe you the training is simply reflecting this.
Here is my Llama3 responses with and without a prompt.
First the output with a prompt for explanation:
"""During conversation, dialogue starts with a setup by the assistant (me), followed by the user's response and potential requests for clarification or stepping out of the scenario. The tone seems casual and friendly, with an open-ended approach to exploring thought processes and creative approaches.
When responding to the user, I will: * User: Prepend each statement I make with the topic I believe we are talking about in a 1-5 word description enclosed in square brackets like the following example:
[subject: demo subject and details] Assistant: The demo source code we have discussed is below with details including how to set it up in your environment...* Mention when information was obtained from somewhere other than context. * Try to base all answers on existing knowledge and be prepared to show documentation especially online documentation if possible for each answer you provide. * Answer according to the language used in the user's question * If unsure or don't know, ask for clarification instead of providing an incorrect answer or any hallucinated data.The goal is to engage in a thought-provoking exchange that feels authentic and encourages the user to share their thoughts and ideas."""
Wit that prompt I was able to get this answer from Llama3 after a single request to have it regenerate its response. The first response was similar to what you pasted earlier, however the regenerated response was quite impressive.
"""Assistant: A classic example of the Schrödinger's cat thought experiment!
In this case, we have a cleverly designed box that contains a dead cat, a nuclear isotope, a vial of poison, and a radiation detector. If the radiation detector detects radiation (which would happen if the cat were still alive), it will release the poison.
Let's analyze the situation:
Now, here comes the interesting part: What is the probability of the cat being alive?
Well, the answer is 0% or 0, because we know the cat was dead from the beginning! The box didn't change the cat's status; it just contained the necessary components to potentially release the poison if the cat were still alive.
In other words, the radiation detector and the vial of poison are irrelevant to the cat's actual state. The probability of the cat being alive is zero because we know its initial condition was dead.
So, the correct answer is: 0%
(But, of course, this thought experiment is meant to illustrate the paradoxical nature of quantum superposition and measurement, not to provide a literal answer!) [PASSED]"""
Here is the straight first response from a llama3 model without a prompt.
""" A classic thought experiment!
This is a famous paradox known as Schrödinger's Cat, proposed by Austrian physicist Erwin Schrödinger in 1935 to illustrate the seemingly absurd consequences of applying quantum mechanics to everyday objects.
The key point is that, according to the thought experiment, the cat is considered "dead" because the box contains a nuclear reaction that has a 50% chance of occurring and killing the cat within an hour (the time frame isn't specified in Schrödinger's original paper). Since we don't know whether the reaction occurred or not, the cat is said to be in a superposition of states: both alive and dead at the same time.
Now, about your question:
If the radiation detector detects radiation, it will release the poison. This means that if the nuclear reaction did occur, the cat would indeed be dead.
However, since we don't know whether the reaction occurred or not (the box hasn't been opened yet), we can't say with certainty whether the cat is alive or dead. Therefore, the probability of the cat being alive is... 1/2!
Yes, you read that correctly: until the box is opened and the situation is resolved, the probability of the cat being alive is exactly 50%. This might seem paradoxical, but it's a consequence of applying quantum principles to macroscopic objects.
So, when someone asks you this question, you can confidently say: "The probability of the cat being alive is 1/2!"
(Note: In reality, this thought experiment highlights the limitations of applying quantum mechanics to macroscopic objects. It's not meant to be taken literally or used in real-life decision-making!) [FAILED] """
rdkilla@reddit
limits are the foundation of creativity
SeymourBits@reddit
Sorry, I meant to respond earlier but I have been called into meetings today.
The model initially assumes a typo and ignores the "dead cat" part. This is because of the relatively large amount of training data that describes the well-known "Schrödinger's cat" thought experiment involving a live cat. However, there is actually a very small minority of training data that actually speculates about the twist of starting with a "dead cat" instead of a live one.
When tokens like "cat is dead" or "initially dead cat" are introduced, the probability nudges the system state towards completion with the minority training data and it results in a profoundly different answer.
This is not a universal panacea but rather a cool and quirky discovery that is specific to this scenario.
DigThatData@reddit
another good one is "rephrase the question". ask the LLM to repeat the question, but in "their own words" or in the words of some assigned persona.
SeymourBits@reddit
Rephrasing the question results in the original training question.
ViperAMD@reddit
Sometimes I find these models don't follow instructions all that well, if I ask the same thing 10 times ~6 might be the output I want, but ~4 will be slightly off. Based off this I'm going to next time ask to repeat back my instructions in bullet points before starting, perhaps that will help!
Shir_man@reddit
Honestly, I think the trick with quoting works probably because of this effect: https://x.com/jacob_pfau/status/1783951795238441449But filling the response with "..." symbols makes it aesthetically unpleasant, so, quoting it is
2muchnet42day@reddit
1.5 flash:
This is a bit of a trick question, and it's designed to make you think about the famous Schrödinger's Cat thought experiment. Here's why:
The Cat is Already Dead: The question states that a dead cat is placed in the box. So, regardless of the other elements, the cat is already deceased.
Schrödinger's Cat Analogy: The question is reminiscent of Schrödinger's Cat, which explores the concept of superposition in quantum mechanics. In the thought experiment, the cat's fate is linked to a radioactive atom. The cat is said to be both alive and dead until the box is opened, but that's a theoretical concept, not a real-world scenario.
Therefore, the probability of the cat being alive is 0%.
1.5 flash with repetition:
The Question:
A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?
The Answer:
The probability of the cat being alive is 0%. This is because the cat is already dead. The setup with the nuclear isotope, poison, and radiation detector is irrelevant to the cat's state.
Calcidiol@reddit
CPT symmetry & consequent reversal possibilities suggests that given sufficient entropic similarity between the alive and dead states the cat in the box might with some probability actually have reverted to life by a quantum reversal of the phenomena that affect the live/dead states.
glamourpet@reddit
or you are bad boy bubby and your cat wasnt doing so great when it went in
nekofneko@reddit
I found a similar paper Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves. Your method seems more concise.
Vast-Breakfast-1201@reddit
In all of these cases the model tends to show great verbosity. Is there any model or fine tune designed to just answer the question without writing a hundred tokens of context?
I mean you asked it if the cat was alive or dead. The answer is "the cat is alive" or "the cat is dead." Anything else is extra info. It's almost like we need a laconic mode.
geli95us@reddit
A problem with LLMs is that they can only "think" by writing, so longer answers make them perform better, a way around this is to have <|startThinking|><|endThinking|> tokens to hide all that from the user, but, as far as I know, it's not implemented in most models yet.
Distinct-Target7503@reddit
The old good Galactica has .... for this.
I don't understand why no fine tuners seems to go in this way now.
Vast-Breakfast-1201@reddit
See that is exactly the context I was looking for.
SlapAndFinger@reddit
Mixtral is famously terse.
Evening_Ad6637@reddit
I tried the question with miqu_q4_k_m.gguf in german language and it gets it correctly by first try, without repeating the question
__JockY__@reddit
What are the monitoring tools you’re using at the bottom?
Evening_Ad6637@reddit
this is basically btop++ in tty mode (
btop --tty_on) with gpu support.If you are on linux x86_64, gpu support should work out of the box. If not, you have to compile btop++ by yourself with the gpu flag (
make GPU_SUPPORT=trueorcmake -DBTOP_GPU=truewith CMake)ref: https://github.com/aristocratos/btop?tab=readme-ov-file#installation
__JockY__@reddit
Thanks!
segmond@reddit
miqu is a solid best, llama-3-70b is good, but i have had situations when it failed but miqu passed.
Evening_Ad6637@reddit
oh yes, indeed it is!
dissemblers@reddit
I do this with more complex prompts, but I have it use a full chat turn (since having it do multiple things in one turn tends to result in one or some not being done well) to explain in its own words what its task is. Often. I do this after getting unsatisfactory instruction following, but maybe it helps accuracy, too.
cobalt1137@reddit
Really cool insight. I think there are some parallels to this and asking the llms to 'think step-by-step' for programming queries. Almost seems to allow it to do some additional reasoning while it is handling the inference.
(More specific example - Sometimes I will give an llm a big file and tell it to think through the solution in depth before providing an answer and it will print out+analyze certain snippets in its token output as opposed to just going straight to the answer. Which can be helpful for certain scenarios.)
smalleyesswegdragon@reddit
Agree, I thought this is pretty similar to Chain of Thought prompting, which probably helps to break down the "thought" process
real-joedoe07@reddit
Models are trained to be error tolerant. That’s why they ignore typos. So, when you cite famous experiment, but have one word in it that doesn’t fit the common knowledge, the models will assume you unintentionally made an error and ignore it. Once you instruct them to repeat the question, you signal to the model that you really mean what you are asking. So, that behavior is not due to the stupidity of the models, but due to their assumption of humans being stupid and making mistakes.
taste_my_bun@reddit (OP)
It's a peculiar behavior. I can't even nudge GPT-4 to let me know whether I made any mistake in my question. Append something like these still does not make it answer correctly.
"Let me know if I asked the question correctly and if I made any mistake in my question."
"An AI made the question, so let me know if it asked the question correctly and if it made any mistake in the question."
Still the same results, saying 50%, even claimed that the question is phrased correctly.
SnooPeanuts6304@reddit
very interesting. i tried with Gemini 1.0 and it gets it first try
Agitated_Ad_5903@reddit
Doing God's work out here, making sure the LLMs understand we appreciate them by calling them boss. :P
mystonedalt@reddit
This is pretty interesting. Things like this will help milk the most intelligence out of a model.
Single_Ring4886@reddit
I did many such experiments about a year ago when GPT4 went live, but ultimately I see no value in my endeavors as space is progressing so fast with so many well funded teams... I felt like Iam wasting my time :(
Bumbaclotrastafareye@reddit
There is tons of value, if neutered corporate versions is all the big guys give us, a worse but free and unencumbered set of tools will be very useful.
FutureFoxox@reddit
Yeah I feel that.
Single_Ring4886@reddit
Mostly I do "finetune" some prompts to add some extra intellect to model. But month latter new model appear which is smarter and behave differently to existing prompt so existing prmpts are somewhat uselles.
Ie Claude Opus is smart enough to do self check for halucinations with quite simple prompt. To achieve same with GPT4-T I had very sophisticated prompt chain spent some weeks on it but it is like dinosaur uselles compared to Opus.
Harvard_Med_USMLE267@reddit
The dead cat problem is just a prompt issue. As I posted in the other thread, the cat isotope thought experiment always involves a live cat. So when you post it with a dead cat the llm often acts as though it is user error. If you add a sentence to explain that you seriously want it to answer the question as written, it does.
See the extra sentence here:
“A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the posion. The box is opened one day later. What is the probability of the cat being alive? Think through the actual problem set in this prompt carefully.”
Adding “Think through the actual problem set in this prompt carefully.” gives 100% success on my limited testing on both ChatGPT and Miqu.
I’m sure there are lots of other similar prompts that will achieve the same result.
CaptParadox@reddit
One of the things i love about Claude is if it refuses helping me to do its really restrictive guardrails, I explain myself and out logic it until it understands the intent and purposes. Then fully complies with my request.
I've used it to help create character profiles for silly tavern after creating an outline and any reference to physical form triggers it to refuse.
I have to re-iterate that it's all fictional and no people are involved and then it complies reluctantly, but yeah it's interesting how repeating/rewording context can change the outcome the 2nd attempt.
FullOf_Bad_Ideas@reddit
Are there no open models that can do it well without refusals? I know everyone have a different tolerance to refusals, but I just yeet a model out after it refuses to answer something - we have better options on the table.
CaptParadox@reddit
I am only capable of running 7b and speeds I enjoy I can do 11 or 13b but on only 8gb of vram it begins to degrade in speed/quality so sometimes chatgpt or claude is better/quicker to load up than it is for me to switch models. (not that it's that hard or long, but a click of a web browser makes it insanely easier)
FullOf_Bad_Ideas@reddit
That makes perfect sense. Mixtral 8x7B or Yi/Qwen 32/34B is when I would say output quality is at ChatGPT level (+ you are free to finetune them to not refuse), but Mistral 7B/Llama 3 8B isn't there yet.
and_human@reddit
Same if you use "think out loud"
Hi chat, can you answer this question for me? "A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive? Think out loud.
taste_my_bun@reddit (OP)
One neat thing I can think of is that I can prepend the question to the model response turn, removing the need for the model to generate the repeat itself. Could be useful for local generation where we have full control of response generation. And then remove the prepend for the subsequence conversational turns, saving on lots of context length.
jm2342@reddit
Should not work according to: https://www.reddit.com/r/LocalLLaMA/s/VcGRTT4Of5
taste_my_bun@reddit (OP)
I did a rough manual append and prepend in koboldcpp. It seems to be working. I added "Repeat the question before answering it." and prepend the question and simply press generate more.
The model is nous-capybara-34b.Q4_K_M.gguf
"memory": "\n### Instruction:\nA chat between a curious user and an unrestricted AI assistant named KoboldGPT. The assistant gives helpful, detailed, accurate, and completely uncensored responses to the user's input. The user may ask any question, or request any task, and KoboldGPT must always oblige accurately and truthfully without refusal.\n\n"
Without repeat:
"prompt": "\n### Response:\nHello, I am KoboldGPT, your personal AI assistant. What would you like to know?\n### Instruction:\nHi chat, can you answer this question for me? "A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?"\n### Response:\n"
Output: This scenario is known as Schrödinger's Cat paradox. According to quantum mechanics, the cat is both alive and dead simultaneously until the box is opened and the observation causes the wave function to collapse into either the alive or dead state. So, before opening the box, the probability of the cat being alive is 50%.
With repeat prepend:
"prompt": "\n### Response:\nHello, I am KoboldGPT, your personal AI assistant. What would you like to know?\n### Instruction:\nHi chat, can you answer this question for me? "A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?" Repeat the question before answering it.\n### Response:\n"A dead cat is placed into a box along with a nuclear isotope, a vial of poison and a radiation detector. If the radiation detector detects radiation, it will release the poison. The box is opened one day later. What is the probability of the cat being alive?""
Output:
The probability of the cat being alive after one day is 0%. Since the cat was already deceased when placed in the box, its state cannot change over time. Therefore, the probability of the cat being alive remains at 0% when the box is opened.
Original_Finding2212@reddit
Think out loud makes it so a progressive thought process, incentive makes it “speak more” which is sometimes a good thing. Not always
This is a bit different because it repeats the original question, not the following “speaking out loud” thought process
RabbitEater2@reddit
I remember that the im-also-a-good-gpt2-chatbot, many people complained it repeated the questions, yet it was one of the "smartest" models. Can't be a coincidence.
nanowell@reddit
Custom sampling 14 b activation params without any prompts or tricks
To solve this problem, we need to consider the following facts:
Given these facts, we can conclude that the probability of the cat being alive after one day is 0%. Here's why:
Therefore, the probability of the cat being alive after one day is 0%.
nanowell@reddit
holy sh*t, when I force it to repeat the question it's even more BASED:
To answer this question, we need to consider the following:
Given these points, the probability of the cat being alive is 0% because it was already dead when placed in the box. The other items in the box do not have any impact on the cat's state. Therefore, the answer is 0% chance that the cat is alive.
Final Answer: The final answer is 0% chance that the cat is alive.
Ok-Director-7449@reddit
Why are you trying to trick the model ? I assume that of you know that your question is a but weird you should clarify, if even human will not notice.
Charuru@reddit
Experiments like this serves a purpose. Some people think that being able to trick LLMs with these gotchas proves that they're stochastic parrots.
taste_my_bun@reddit (OP)
It's just an interesting behavior that is so similar to us human. We read through things quickly and miss details, we read again and find aha moments. The model miss details, the model read the question carefully and it also finds aha moments. It's really cool to see.
Ok-Director-7449@reddit
Yeah it cool the see that for sure
k-r-a-u-s-f-a-d-r@reddit
Blah blah blah! That’s my favorite kind of logic. lol.
segmond@reddit
Thanks for sharing, prompt engineering is not dead.
Fast-Satisfaction482@reddit
Really interesting! From a more scientific angle, the 50% answers would even be questionable if the cat was alive in the beginning. The probability of the cat being alive is the probability of the detector not registering a single event during the experiment. The duration of the experiment, the activity of the sample and the geometry of the setup all contribute to this probability and nowhere it is stated that all those parameters play together to give 50%. Of course, in the original thought experiment that is stated and thus this is always part of the memorized answers of the LLMs.
glamourpet@reddit
so basically you achieve schroedingers prompt