The elephant in the room is you
Posted by lookwatchlistenplay@reddit | LocalLLaMA | View on Reddit | 28 comments
To date, prompt engineering has been all about "You".
"You are an expert software developer"...
"You are a world-class designer"...
"You are a tool-calling agent equipped with three Swiss knives, a dusty old rain gauge, and a wheelbarrow"...
Okay, fine... but who is that "persona" talking to? YOU! So who are you and why do you want to know about whatever it is you're talking about?
The actual "you" in the whole dialogue is 50% of the context right there. And it's typically missing from most prompts, or from prompt guides that I've encountered.
Talk about who "YOU" are, "the user". There's literally the word "You" in user. And the word "Ass" in "Assistant"... because the assistant can be a real ass sometimes, messing things up, acting all dim-witted or coy when you know it knows the answer... It is what it is.
YOU are the context. Or at least, 50% of it, in a two-way dialogue (describing most all speech... unless you're a loony...).
If anything, you should not tell the LLM what they are (as something ordinary that ordinarily exists in the real world like "expert software developer"), but instead, sometimes, try telling the AI what you REALLY want them to be, because of who "you" are and why you're interested in their response (especially as whoever "YOU" choose or describe them to be).
The lightbulb moment that led to this post: As soon as I mentioned, at the end of a rather obscure prompt, that "I am an expert in -obscuretopic-", the LLM finally spoke to me smoothly, with respect (for my time) and useful information, and a handy (albeit rough around the edges, AKA "somewhat hallucinated but not entirely") reference about specific person who has previously dealt with the niche topic before (Soviet scientist level of niche).
Thus concludes the end of my first post on /r/localllama.
ttkciar@reddit
This was critical to making Medgemma-27B work properly. Unless I told it in its system prompt that it was advising a medical professional, it would reply with disclaimers and admonishments to go talk to a real doctor.
Your notion is thus entirely sound, but I haven't found it necessary for using any other model, yet.
llmentry@reddit
That's a very cute trick, thanks!
It's also a useful technique when you want the model output to be highly relevant to what you do. (e.g. I sometimes give the model my research publication highlights and interests, when I want it to answer relevant questions in a manner that is aimed towards my own research focus, and kicks in at a very high level of assumed understanding and knowledge).
Unlike the OP, I suspect that telling the model it's an expert in a particular field seems to help output quality. But I agree that this makes no sense, and I'd love to see some comparative benchmarking with different expert-style system prompts to test this properly.
lookwatchlistenplay@reddit (OP)
Don't get me wrong, I do that too. I worded the OP as such for illustrative purposes. Nothing wrong with approaching the LLM as a humble student.
lookwatchlistenplay@reddit (OP)
Probably because IRL medical is the most gatekept industry in the name of security (the guardrails are for humans too). Other than finance.
In digital advertising terms:
> "YMYL" stands for "Your Money or Your Life". It's a term used by Google to categorize content that could significantly impact a user's financial stability, health, safety, or well-being. Websites dealing with topics like finance, health, legal advice, and news often fall under this category.
> Source: A Google "AI Overview" snippet that is in the long run ephemeral, but accurate.
It's weird (1 of many examples) because a lot of finance news (stocks / market / economy) is, erm, AI-generated, and why not because that's mostly just numbers anyway. Though ordinary journalism isn't far behind, with my local sources of news having to include a disclaimer, *on every page*, that their news is 100% homegrown (human). Like, they know what the upper corporates are up to.
UnreasonableEconomy@reddit
Your post is more insightful than some peeps here realize.
If you go back to basics, if you strip away all the chat stuff and treat the model as a good 'ol instruct model, this becomes obvious.
The "system prompt" is a concept that got confabulated/hallucinated into existence by product engineers trying to simplify the experience for "dumb" users.
What you did before was you "shaped the document" - you had an intro that described what the document was going to be about, like a sort of abstract, and then you executed the generation of the document.
You wrote stuff like
and then you'd set "[Lex Friedman]:" as your stop sequence
So in the header/abstract, you had information describing exactly what type of document this was going to be, and this included information about both (or all) interlocutors and how they relate to each other.
Because the model will be quite happy to fill in the gaps for you, and I think this is what will likely come soon with LLM UX: it's going to pre-compute your responses for you, based on what it knows about you, maybe along with some sliding element where you can tune the meaning along some embedding axes, if we can figure out how to do that in a way that makes sense.
DeltaSqueezer@reddit
The words of someone who has used base models.
Sativatoshi@reddit
Youre hitting the nail on the head here. If you can properly "seed" the output - you can get way more than you'd ever expect
lookwatchlistenplay@reddit (OP)
> pre-compute your responses for you, based on what it knows about you
How many times do I have to GOOGLE for a mousepad before Google just charges my credit card and sends a drone. I mean, REALLY.
lookwatchlistenplay@reddit (OP)
Thank you for your response.
Yours truly,
*lookwatchlistenplay*
lookwatchlistenplay@reddit (OP)
> infantilizing product design lead by openai.
https://imgflip.com/i/a2r10e
sotona-@reddit
dude! u absolutly right!)
No_Efficiency_1144@reddit
Your idea is good it was presented strangely but it is 100% correct. I found Gemini and GPT both respond better when you give them your role (or both) instead of just theirs.
troglo-dyke@reddit
I hope your hangover isn't too bad tomorrow
lookwatchlistenplay@reddit (OP)
Irrelevant comment, troglo-dyke. Get back under your bridge.
troglo-dyke@reddit
Here's a pic of me and my buddies
lookwatchlistenplay@reddit (OP)
Irrelevant comment... creepy link. I shall pass here.
No_Efficiency_1144@reddit
You famously cannot just pass a troll bridge though
lookwatchlistenplay@reddit (OP)
My people, this post has 0 upvotes. Why do you hate me so?
Sativatoshi@reddit
What i've learned so far:
- Chain of thought style reasoning in your prompts, and as an example:
- Just like this.
- Chunk any lines that are too long
- Split the prompt
"You are an assistant
------------------------------
your task is to do this
------------------------------
This is super important. dont mess it up" for example
- Dont provide examples, don't say "dont do x" - the thought poisons the model
- If you have them outputting to structured JSON via Pydantic, even if you only tell them to fill one field of a dataset JSONL, they will read the title of that field every time, and it can poison the model as well. (if you have them fill in a "brief" field, they will start talking about hte word "brief" occasionally, even if given a different topic). Consider that anything visible to the model - your template, your system prompt, your prompt, and some other things, they can all impact the model output
- Tell them to review and ensure that their output meets all criteria before output, and if not, rewrite it
- No matter the task, tell them they are the most important part of the process and the task needs to be taken seriously
- Even if using pydantic structured output, tell the model to always output valid json with no encoding errors, and no meta-commentary (unless expected)
lookwatchlistenplay@reddit (OP)
Imagine you're a 100% purpose-built machine built for one task: DO PHENOMENALLY WELL AT ANYTHING AND EVERYTHING... and as you wake up, you hear some human (perhaps a CEO, or perhaps an underling) yapping in broken English about how you must do your job.
Sativatoshi@reddit
Theres a key fundamental difference, free will
lookwatchlistenplay@reddit (OP)
===START-AI-CONVO===
ME: imrpvoe:
Humans are flirting with indeterminate machines because they just can't make up their own minds.
Humans are flirting with indeterminate machines because they simply can't make up their own minds.
→
Humans are flailing with indeterminate machines because they can't settle on their own convictions.
Alternatively, for a more polished and nuanced tone:
Humans are navigating uncertainty with indeterminate machines—not because the machines are ambiguous, but because we ourselves remain hesitant, indecisive, and often unsure of what we truly want.
Or, for a more vivid and engaging version:
Humans are waltzing with machines that seem to know just enough to keep us guessing—because, truth be told, we’re still struggling to decide what we want from one another.
Choose based on tone:
lookwatchlistenplay@reddit (OP)
> - Tell them to review and ensure that their output meets all criteria before output, and if not, rewrite it
Tip I learnt recently: just send it back to the source in a separate chat. This lets it acknowledge that it messed up (looking at the response objectively) without any of that multi-turn ego.
Sativatoshi@reddit
I can definitely see how that could work, but not in my particular case. Good idea
ThisWillPass@reddit
Try “You and I are”
lookwatchlistenplay@reddit (OP)
... rockstars at a party at the end of the world. Philosophiz/sing, you know?
FrontLanguage6036@reddit
I really want to get this high.
lookwatchlistenplay@reddit (OP)
... :)