The most useful AI work I see now is not chat. It is boring background stuff
Posted by Ryannnnnnnnnnnnnnnh@reddit | LocalLLaMA | View on Reddit | 46 comments
for me the exciting part is not another chat UI anymore.
the really useful stuff is much more boring
classification
routing
ranking
cleaning messy inputs
watching a stream of text and surfacing the few things that actually matter
that is where AI starts to feel real to me
not when it gives a fun answer in a demo
when it quietly removes manual work every day
I keep feeling local models fit that kind of job better than people admit
always on
cheap to run once set up
private by default
good enough for a lot of narrow tasks
I think a lot of people still judge local AI by asking can it replace ChatGPT for me
for actual products and workflows, I care way more about whether it can sit in the background and make a messy system usable
maybe I am wrong, but local AI gets more valuable the less it looks like a chatbot
jacek2023@reddit
Looks like new spam is coming, lots of upvotes and fresh account :)
Worldly_Expression43@reddit
these fucking idiots think typing in lower case will make their slop sound less chatgpt-y when it's still the same awful writing style
typical-predditor@reddit
All of that boring chore work like karma farming on fresh accounts.
JuliaMakesIt@reddit
Exactly.
Especially with local models, the ability to have near-human level classification, clean up, summarization and analysis is amazing. And the speed and volume of data a local LLM can process is a godsend for anyone working with large messy datasets.
The fact that you can run these models on consumer hardware, and without having to send proprietary data to third parties is remarkable.
techno156@reddit
I'd disagree on the clean-up front. For things like cleaning up data, conventional tools tend to be faster, predictable, and more efficient.
You don't need to worry about context limits when using the stream editor, for example, since it can work with arbitrarily large amounts of data, and it requires significantly less compute power.
JuliaMakesIt@reddit
I agree with you. Raw LLM’s aren’t a great fit and require a lot of babysitting.
I use an agentic framework (the Qwen_Agent Python module) with tool calling. The LLM orchestrated, calling tools as needed to process the various components of the data and sanity checks output using validation tools.
Tooling is doing the heavy lifting, but the LLM orchestration makes it scale nicely and turns a monotonous manual multistep effort into an essentially unattended one.
havnar-@reddit
The amount of posts I see saying “I made this new tool” that just ends up being a vibe coded wrapper around a call to an agent is shocking.
Borkato@reddit
I’ve been making my own wrapper for personal use and it’s harder than it seems :( either that or I’m just stupid. It’s fun trying to figure out how to make it print and behave exactly like I want though.
PM__ME__YOUR__PC@reddit
import chatgpt; apikey = "***"; chatgpt.ask(question, apikey);
Borkato@reddit
Where’s the history, the commands, resuming old chats, the tools, the fixes for the tools, the support for multiple models and their quirks, viewing chat history in a compact layout, debugging, file creation, tool handling, the system prompt…
Nelson-Bolt@reddit
You make it sound harder than it actually is, all of this can be done in 500-1000 lines of python or node js. and why did you throw "file creation" and "system prompt" there like they are a technical challenges.... unless you are not a developer all of this is trivial, there are thousand AI wrappers on github that implement all of this and more.
Borkato@reddit
I can tell you’ve never actually made a wrapper that does anything useful.
Nelson-Bolt@reddit
I made a JavaFX (not web) interface for the translategemma models to translate entire novel volumes, I implemented multiple tools and made a context management system to preserve story details and session information, and used multiple agents at the same time to speed it up.
But I think my opinion doesn’t count compared to your 1000 ChatGPT clone interface.
Borkato@reddit
That’s cool! I don’t care.
Nelson-Bolt@reddit
hahaha
Borkato@reddit
Hehehe
havnar-@reddit
It’s on github, make a pr
RoomyRoots@reddit
Truly the webdev moment of AI.
waiting_for_zban@reddit
I have to partially agree with a very big astericks. You still cannot trust the output. I have done all the above on many projects, it is still "stochastic". It's getting better. But I still have review and babysit. In comparison, training a BERT NLP classifier is much more streamlined, and I can run it nearly blind when it comes to trust. I am yet to finetune an LLM for such tasks though.
typical-predditor@reddit
Cleaning up messy inputs should be done via rigid scripts. Chew through the inputs until exception falls through the cracks. AI examines the input and suggests a new rule for the script. Then you keep going.
Moreh@reddit
in an ideal world maybe but much of the time for different use cases more economical to throw it at llms
Ps3Dave@reddit
But a script uses a tiny fraction of the power that a LLM uses. It's better to have a LLM write a script once and then have that script running 24/7.
havnar-@reddit
This paradigm is lost on many vibe coders. But soon, when frontier subs will become expensive, that will probably become a teachable moment
Moreh@reddit
Yeah totally, if you can write it once. That is not always the case, though. To get an LLM to write a script for all possible cleaning operations, you still need to make the LLM look through every single record to understand the rules it must create. And if the data changes and you need new rules, you still have to run the LLM on those new records anyway
Borkato@reddit
How easy/hard is it to get started with bert nlp? I’ve never heard of it and want to try if it’s that good :)
waiting_for_zban@reddit
It depends on your pre-existing knowledge. But tbh, hugginface has done lots of amazing work on documenting nearly everything NLP related. They have lots of tutorials on how to do so. If it's too daunting I think youtube videos can help. My 2 cents, the most important part is data though. You need to have good quality data.
Borkato@reddit
Thank you!! Adding this to my list of stuff to check out
Suitable_Extent1960@reddit
youre spot on about the boring stuff being where the real value is
the chat demos are fun but theyre a distraction from the actual work
companies have decades of messy processes and data locked in weird systems
an AI that can just sit there and clean, sort, and route that garbage without needing a prompt is a game changer
local models make sense for this
they run offline, dont leak data, and once you set them up the cost is basically zero
the benchmark shouldnt be can it write a poem, it should be can it run unattended for six months
if your AI needs a chat interface to function, its probably not doing useful work
its about reliability, not creativity
soc2 compliance and audit trails matter way more than a clever response
everyone wants an AI employee but they really just need an AI janitor for their data
the hype is about replacement, but the money is in making existing broken systems slightly less broken
thats not as sexy but it actually works
edit: wording
Worldly_Expression43@reddit
ai slop
Worldly_Expression43@reddit
just because you type in lower case doesn't make it less obviously this is ai slop
Sangkwun@reddit
tried setting up something like this for monitoring a few niche tech blogs last year. the model part worked. the part that broke it was maintenance. three feeds changed their structure, two went behind paywalls, one started injecting sponsored content that confused the classifier. spent more time patching the pipeline than actually reading the output. eventually just killed the whole thing.
a_beautiful_rhind@reddit
I think the biggest use is spamming, clearly.
Ok_Mammoth589@reddit
And it's such a johnny-come-lately thing to spam. This has been parroted for two years now. That agents are coming. Chat was only the beginning, blah blah blah.
Joozio@reddit
This exactly. I run a 35B model on a Mac Mini M4 doing classification, routing, and triage on background streams. The compounding value comes from it running while I sleep, not from any single clever prompt. Surprisingly hard to explain to people who haven't built it - the boring silent stuff accumulates fast.
SkyFeistyLlama8@reddit
It works as a second brain. I can dump a bunch of random research PDFs and personal reading files into a single temp folder, get Qwen 35B or Gemma 26B to iterate over those files to create a quick summary, and then the LLM suggests where to place those files.
I'm not brave enough to do tool calling with actual filesystem calls so I do the file moves by hand.
MaCl0wSt@reddit
what I did recently was make a tool in openwebui for the model to specifically only be able to write a .md note in a subfolder in my obsidian vault, it cant do anything other than create notes in that subfolder. having that folder synced with syncthing across my devices, its a pretty sweet experiment so far.
themule71@reddit
You can have your harness SW run in a container, with a single working directory mounted as a volume. Backup it frequently, in case the AI hallucinates and deletes everything. You can have multiple volumes, eg you can have a read-only one for source documents, which the AI won't be able to delete. If feeling fancy there are filesystems with revision control built in.
Slycen-Dice@reddit
You're hitting on something most people miss. The chat interface is just a UI, the real value is in something that actually knows your context, remembers what you were working on last week, and handles the repetitive stuff without you having to re-explain everything from scratch. That's where the leverage actually is. Most tools treat each conversation like it's happening in a vacuum, which defeats the purpose of having an AI that's supposed to make you more productive over time.
1EvilSexyGenius@reddit
This is expected. It's only the beginning. The fun comes when regular people start slapping small things together to create bigger cooler things 🪩 Things we don't even know we need yet
RoomyRoots@reddit
Chat has always been the worst way to use AI in my opinion, well, that and "art". It's quite a sad state that we haven't found many better uses for it than Watson was already doing a decade ago,
Ambitious-Hornet-841@reddit
agree. Which local models have you found best for that ‘background cleaning/routing’ work? And what’s one small task where local AI surprised you by outperforming a cloud API?
danigoncalves@reddit
AI enables the automation of tasks that were previously difficult to automate. It can also be seen as an 'inverted AI pair programmer.' In a typical pair programming session, one person writes the code while the other provides ideas and feedback. With an AI partner, this dynamic is reversed: you, the developer, 'drive' the process by guiding the AI, which then writes the code. It also writes english and structured text way better than me :D
TwistedManiac1@reddit
True, hope this mindset goes into non tech people as well, the value of local AI is immense
WillemDaFo@reddit
True. Sadly, what you are describing is Palantir, not less daily grind for the working class.
cjami@reddit
I keep telling people that LLMs are great natural language processors. Little intuitive hops like turning a sentence into instructions, extracting structured content, or judging content on a strict criteria, opens up lots of possibilities in automating processes
It trivialises a lot of problems that would have been terrible to implement in code.
ttkciar@reddit
It always was!
The only useful chat application I ever saw was tech support.