Anyone running agents 24/7, not just in sessions?
Posted by Music_is_ma_soul@reddit | LocalLLaMA | View on Reddit | 8 comments
I've got llama3 70B running on a dual 3090 setup through Ollama. Built a python script that checks financial data every morning, analyzes it, and sends me a summary on Telegram.
Problem is it's basically a cron job with amnesia. Every run starts from scratch. It told me the same "AAPL is showing unusual volume" insight three days in a row because it doesn't remember what it already told me.
I hacked together a SQLite log which stores the last 10 summaries into the prompt as context but that's already getting long and I know it won't scale past a few weeks. I'm thinking of doing a markdown file for short term and keeping the sql as a dbish??
Anyone here actually have an agent running long-term that remembers previous runs? How are you handling the memory? Just curious what setups people have landed on.
Awkward-Boat1922@reddit
Maybe use a 'better' model.
Music_is_ma_soul@reddit (OP)
Dude a model won't change the memory problem.... agents by definition aren't just "better" models
jacek2023@reddit
learn about agentic workflows, by working on code you store documents into .md files, same mechanism can be used for other things, like your case
Music_is_ma_soul@reddit (OP)
I'm already doing the md file approach and it works well short term. My question is more about what happens after months of daily runs. Like if the agent has been running every day for 90 days, are you feeding the whole file back in each time? I'm wondering if people who've run agents longer term have found good ways to handle that or if most folks just aren't running them that long.
Appreciate the suggestion though, definitely on the right track with the file based approach.
DinoAmino@reddit
Your problem is not well defined. You said it was telling you the same thing every day. What is it supposed to be telling you? Is it supposed to omit anything it already said over the course of many weeks? This probably isn't a storage problem.
Music_is_ma_soul@reddit (OP)
Fair point, let me be more specific. The agent checks AAPL price, volume, and news every morning. What I want is for it to say something like "volume spiked 40% today, this is the third time this month that's happened and the last two times the price dropped within 48 hours." Instead it just says "volume is unusually high today" because it has no idea what it said yesterday or what happened last week. The goal is to have the agent build up context over time so its analysis actually gets smarter, like a human analyst would by remembering patterns across weeks. Right now I dump the last 10 run summaries into the prompt but after a few weeks that's going to blow past the context window. And even if I cram it all in, the model will probably start ignoring stuff in the middle anyway. Im thinking of something like how our brain works, short term memory for patterns (markdown?) and long term memory for facts (sql?). Trying to get a sense if someone has tried it before i go this long winding road
DinoAmino@reddit
Summarization and analysis are two different tasks. Analyzing daily summaries is probably not the best way to do it.
hawseepoo@reddit
Personally, I would rework the project a little bit. Keep the summaries in the SQLite database, but allow the model to write a query to summarize it instead of including 10 entries in the context. Tho I can’t know how well this would really work unless you show the DB schema