Any LLM projects with contextual awareness?

Posted by EternalOptimister@reddit | LocalLLaMA | View on Reddit | 7 comments

Every time I consider an “AI assistant”, I immediately encounter (simple to fix) situational awareness problems. I need the model to know that today is the 31st of December 2024, HH:MM:SS! That would be the very least. On the other hand - in most cases - I need it to know who I am, where I live, what my profession is, what I am working on and so much more. So question is: are there any companies working on assistants on steroids with actual situational awareness? Would be feasible to make, but of course having to provide all this info as (if possible permanently cached) prompt would increase the cost.

Reply to Post

7 Comments

[-]

ComprehensiveBird317@reddit

Tell a good coding agent to create a proxy to your LLM provider and add those information to the system message automatically.

[-]

SomeOddCodeGuy@reddit

>I need the model to know that today is the 31st of December 2024, HH:MM:SS! That would be the very least. I added this to Wilmer and it went really poorly. The system timestamped every message, including the current one coming in, and Mistral Large 123b absolutely made a mess out of tracking those timestamps. I ended up just disabling the whole feature so I could rethink how to do it. Tokenization is a big problem here. Someone had recommended to me after that I should instead have the program determine the difference in time between the last message and the current, and instead of giving timestamps just say "It has been x hours since the last message" in plain english, so that maybe the model would better understand that. For now, I'm not trying anything that number heavy again.

[-]

Everlier@reddit

Open WebUI has a memory feature allowing to specify custom pieces of information like these. Additionally, it allows defining system prompts for permanent things (and some variables like date/time).

[-]

Herr_Drosselmeyer@reddit

This is solvable with front-ends that update the context via macros. I use SillyTavern but there are certainly others. For your needs, you'd just write a simple character card (essentially a prompt) along the lines of: >It is {{date}} and the current time is {{time}}. You are a capable AI assistant to {{user}} who works as a XYZ for company ABC \[etc.\] SillyTavern also allows for a user "persona" which would be independent of the character you're using. So say you define different types of assistant to attempt to focus them in a direction, give them a specific demeanor (professional vs jovial perhaps) etc. You then would only need to update the user "persona" for all the different characters to know what you're up to.

[-]

EternalOptimister@reddit (OP)

I didn’t know sillytavern supported semi dynamic prompts. Do you happen to know if openwebui support something similar? (I have an instance of the latter running). Could basically have it know the calendar info from today and my static information (in each prompt). Everyday have it summerize the daily information, then weekly/monthly/yearly as time passes and have the info aggregated in separate prompts and use different persona’s with access to different levels of detail as needed for the question. Then have a prompt analyser that decides which persona would be a good fit to answer the current prompt… but all this would require a sophisticated and intuitive system that operates together. Hence the discussion

[-]

cobbleplox@reddit

I experimented with that stuff some time ago. You don't really need anything special model-wise. It's all a matter of context management to inject some time information in smart ways (easy to understand, not blowing up context size too much and not killing the cache all the time). Regarding the personal information, that seems very trivial since it is static information. You should be able to have that as part of your system prompt. So that should be pretty easy with just ollama. It supports making your own model variants by just combining them with a custom system prompt.

[-]

Ylsid@reddit

You would need to do a lot of training of some kind for this If you don't want to embed it into a prompt you're better off waiting for some new breakthrough