LLMs are a 400-year-long confidence trick
Posted by SwoopsFromAbove@reddit | programming | View on Reddit | 362 comments
LLMs are an incredibly powerful tool, that do amazing things. But even so, they aren’t as fantastical as their creators would have you believe.
I wrote this up because I was trying to get my head around why people are so happy to believe the answers LLMs produce, despite it being common knowledge that they hallucinate frequently.
Why are we happy living with this cognitive dissonance? How do so many companies plan to rely on a tool that is, by design, not reliable?
Distinct-Shoulder592@reddit
okay so , I think of LLM systems like a loop that keeps getting smarter. A Hermes-style agent handles quick decisions in the moment, while an LLM Wiki Compiler turns those actions into organized knowledge that keeps building and improving over time.
robhaswell@reddit
Not common knowledge, not even nearly. Your average retail user MAY have read the warning "AIs can make mistakes" but without knowing how they work I'd say it's difficult to understand the ways in which they can be wrong. You see this on posts to r/singularity, r/cursor etc all the time.
nomorebuttsplz@reddit
Not common knowledge, I agree. Not even knowledge.
Intrepid-Stand-8540@reddit
Yeah. Everyone I've talked to IRL that is not a programmer, thinks AI is always correct. Very scary.
ConceptJunkie@reddit
I subscribed to to r/singularity briefly, but it mostly seemed like a cult for dumb people.
EnchantedSalvia@reddit
It was a dumb place a year ago but got better recently as more reasonable and moderate voices entered, all the crazies went to r/accelerate
robhaswell@reddit
You can have an upvote for that
kappapolls@reddit
what do you mean they aren't as fantastical? just a few weeks ago some dudes used GPT 5.2 to solve a couple erdos problems and formalized the proofs in lean. that's pretty much sci-fi fantasy come to life
dummytroll@reddit
feels like there's a lot of denial in this subreddit, people refusing to accept new realities, i wonder how much time most of these people have tried a recent LLM properly.
claytonkb@reddit
I have access to frontier models through my employer for my day job (engineering). They are being M.A.S.S.I.V.E.L.Y. overblown. "Hype" doesn't even begin to describe it.
dummytroll@reddit
Curious which frontier model you're referring to, that's this disappointing.
claytonkb@reddit
I have access to ALL of them through the license. They all have the same problems.
First, let me note that current-gen is already useful. I save probably 4-8 hours per week using it, which is enormous. But my main mode of usage is as a glorified-RTFM-bot, and as a boilerplate typist. As for writing something I don't already know how to do (in principle), I would never trust current-gen AI to do that.
You've heard the term "prompt engineering", no? The mere need for "prompt engineering" is an automatic rebuttal of any claim that these systems are "generally" intelligent under any reasonable construction of general intelligence. If the system was so intelligent, it would not need to be "prompt engineered", it would just ask the right questions and "prompt-engineer" itself.
Pre-training-based AI cannot, by definition, be generally intelligent. Can it form an ingredient of a generally intelligent system? Sure. But that's an entirely different question, because it is only the pre-training component which is subject to the mythical "exponential scaling law".
As for the problems I see in how current-gen AI "thinks", I think this video gives a good start on how I view the issue, so maybe quickly scan through that if you want to understand where I'm coming from. What people expect AI to be able to do is itself irrational.
To add to this (yes I've thought about this a lot), suppose I live somewhere that I've never seen any kind of boat. I've decided I want to paint the best boat ever. So I tell the AI, "Draw me the best boat ever". But I've never seen a boat, so how in the world will I know whether the result generated is actually good, or just crap? I can't know that. The AI could walk me through a visual guide of the types of boats, and so on, but the point is that I don't actually have anything to contribute to this area and a sufficiently intelligent AI would just say that. Note that this is precisely the same problem that educators face when dealing with the ambitions of young students -- they often dream of things that they don't even know for sure what it is they're dreaming of. In itself, that's not bad or wrong, it's just that that dreaming requires guidance and refinement in order to be pointed in a direction which has some chance of actually culminating in a real fulfillment of whatever it is that this child is dreaming. The educator is vastly more intelligent than the child in the sense of overall understanding of the world, and he or she uses that vastly superior intelligence to challenge the child, not validate them where they are at, precisely because the educator knows that the child faces a long ramp of growth ahead of them before they will even be in a position to meaningfully dream of the particular thing that they are now only vaguely dreaming of. AI doesn't even comprehend these basic issues, instead, it is productized and RL'd to "maximize screen time" just like every other corporate-tech product out there.
ffiarpg@reddit
Because even if it's right 95% of the time, that's a lot of code a human doesn't have to write. People aren't reliable either, but if you have more reliable developers using LLMs and correcting errors they will produce far more code than they would without it.
Valmar33@reddit
The difference is that if you didn't write the code ~ debugging it will be a total nightmare.
If you wrote it ~ then at least you have a framework of it in your mind. Debugging it will be far less painful, because you wrote it with your mental frameworks.
Reliable developers statistically get no meaningful benefit from LLMs ~ LLMs just slow experienced devs down as they have to spend more time debugging the code the LLM pumps out than if they just wrote it from scratch.
Tolopono@reddit
Andrej Karpathy: I think congrats again to OpenAI for cooking with GPT-5 Pro. This is the third time I've struggled on something complex/gnarly for an hour on and off with CC, then 5 Pro goes off for 10 minutes and comes back with code that works out of the box. I had CC read the 5 Pro version and it wrote up 2 paragraphs admiring it (very wholesome). If you're not giving it your hardest problems you're probably missing out. https://xcancel.com/karpathy/status/1964020416139448359
Opus 4.5 is very good. People who aren’t keeping up even over the last 30 days already have a deprecated world view on this topic. https://xcancel.com/karpathy/status/2004621825180139522?s=20
Response by spacecraft engineer at Varda Space and Co-Founder of Cosine Additive (acquired by GE): Skills feel the least durable they've ever been. The half life keeps shortening. I'm not sure whether this is exciting or terrifying. https://xcancel.com/andrewmccalip/status/2004985887927726084?s=20
I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents, subagents, their prompts, contexts, memory, modes, permissions, tools, plugins, skills, hooks, MCP, LSP, slash commands, workflows, IDE integrations, and a need to build an all-encompassing mental model for strengths and pitfalls of fundamentally stochastic, fallible, unintelligible and changing entities suddenly intermingled with what used to be good old fashioned engineering. Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession. Roll up your sleeves to not fall behind. https://xcancel.com/karpathy/status/2004607146781278521?s=20
Creator of Tailwind CSS in response: The people who don't feel this way are the ones who are fucked, honestly. https://xcancel.com/adamwathan/status/2004722869658349796
Stanford CS PhD with almost 20k citations: I think this is right. I am not sold on AGI claims, but LLM guided programming is probably the biggest shift in software engineering in several decades, maybe since the advent of compilers. As an open source maintainer of @deep_chem, the deluge of low effort PRs is difficult to handle. We need better automatic verification tooling https://xcancel.com/rbhar90/status/2004644406411100641
In October 2025, he called AI code slop https://www.itpro.com/technology/artificial-intelligence/agentic-ai-hype-openai-andrej-karpathy
“They’re cognitively lacking and it’s just not working,” he told host Dwarkesh Patel. “It will take about a decade to work through all of those issues.”
“I feel like the industry is making too big of a jump and is trying to pretend like this is amazing, and it’s not. It’s slop”.
Creator of Vue JS and Vite, Evan You, "Gemini 2.5 pro is really really good." https://xcancel.com/youyuxi/status/1910509965208674701
Creator of Ruby on Rails + Omarchy:
Opus, Gemini 3, and MiniMax M2.1 are the first models I've thrown at major code bases like Rails and Basecamp where I've been genuinely impressed. By no means perfect, and you couldn't just let them vibe, but the speed-up is now undeniable. I still love to write code by hand, but you're cheating yourself if you don't at least have a look at what the frontier is like at the moment. This is an incredible time to be alive and to be into computers. https://xcancel.com/dhh/status/2004963782662250914
I used it for the latest Rails.app.creds feature to flesh things out. Used it to find a Rails regression with IRB in Basecamp. Used it to flesh out some agent API adapters. I've tried most of the Claude models, and Opus 4.5 feels substantially different to me. It jumped from "this is neat" to "damn I can actually use this". https://xcancel.com/dhh/status/2004977654852956359
Claude 4.5 Opus with Claude Code been one of the models that have impressed me the most. It found a tricky Rails regression with some wild and quick inquiries into Ruby innards. https://xcancel.com/dhh/status/2004965767113023581?s=20
He’s not just hyping AI: pure vibe coding remains an aspirational dream for professional work for me, for now. Supervised collaboration, though, is here today. I've worked alongside agents to fix small bugs, finish substantial features, and get several drafts on major new initiatives. The paradigm shift finally feels real. Now, it all depends on what you're working on, and what your expectations are. The hype train keeps accelerating, and if you bought the pitch that we're five minutes away from putting all professional programmers out of a job, you'll be disappointed. I'm nowhere close to the claims of having agents write 90%+ of the code, as I see some boast about online. I don't know what code they're writing to hit those rates, but that's way off what I'm able to achieve, if I hold the line on quality and cohesion. https://world.hey.com/dhh/promoting-ai-agents-3ee04945
Valmar33@reddit
Saying words on the internet is easy. Marketing and hype is easy.
But the reality is that LLMs simply fall apart when trying to do anything but very simple tasks they have been repetitively trained on with many, many examples written by real people.
The model collapse problem is also a very major issue ~ LLMs only function when fed real stuff by real people. LLMs fed LLM-generated stuff fall apart very quickly. And as more stuff is AI-generated, LLMs will inevitably fall into that trap more and more.
https://www.youtube.com/watch?v=lV29EASsoUY
zacker150@reddit
I think the model collapse problem is largely overblown for two reasons:
Valmar33@reddit
Do you understand how model collapses work? They happen when an LLM is fed LLM-generated data. It is based on how LLMs process and tokenize text.
Text output will tend towards as statistical mean over time, due to the peculiar oddities around how LLMs produce output by choosing the statistically next sets of words or phrases based on what LLMs have been trained on.
Random variation doesn't prevent this, because the random variation itself relies on statistical probabilities built into the algorithm. There is a tendency for the algorithm to choose the more statistically-probable next tokens, rather than the outliers.
Therefore, LLMs fed LLM-produced data will tend more and more towards the mean, because outliers are getting cut out more and more with each generation.
This is a problem inherent in any model that relies on statistical probabilities. Meanwhile, humans in reality do not "predict" next sets of words or phrases. We choose our words based on their semantics ~ what words will convey the meaning that we intend.
LLMs, on the other hand, are purely syntax-driven ~ what tokens are statistically related to other tokens. For this, they need real data from real humans beings in order to provide novelty and coherency. But as we run out of real data that isn't LLM-produced, due to the massive influx of LLM-produced text and data on the internet, LLMs will inevitably begin consuming LLM-generated content, slowly tending towards a model collapse.
Tolopono@reddit
Anyway, here it is winning gold in the imo https://intuitionlabs.ai/articles/ai-reasoning-math-olympiad-imo
Valmar33@reddit
A meaningless "award" because LLMs are still fundamentally no different than they've ever been.
Tolopono@reddit
Lmao
Valmar33@reddit
Which is meaningless, because an LLM can just be programmed to do stuff. There's no skill, talent or creativity involved.
Tolopono@reddit
Whatever helps you sleep at night buddy
Valmar33@reddit
Well, I am not surprised that a true believer in LLMs is defending LLMs.
Tolopono@reddit
Hard evidence makes believers out of anyone. Or at least anyone rational
Valmar33@reddit
lmao, LLMs can be programmed to spit out any sort of text you want if you train the model well. It's why any sort of "award" for an LLM is pure, overhyped nonsense. Like all LLMs of today.
Again, no intelligence, creativity, skill or talent required ~ just training data to make a model spit out results you want.
Tolopono@reddit
What if the result i want wins the gold in the imo
Or writes a novel so good that it won a grand prize and the peoples choice award before getting caught https://automaton-media.com/en/news/ai-generated-isekai-novel-that-won-a-literary-contest-grand-prize-and-readers-choice-award-has-its-book-publication-and-manga-adaptation-cancelled
Valmar33@reddit
LLMs are not "writing" novels ~ the designers plagiarize other people's works and conglomerate that into something that merely appears to be good, because it was trained on certain sources.
There's no skill, no creativity, no talent, nothing. Just plagiarizing.
Tolopono@reddit
Plagiarism means substantial similarity between two works. Which novel was plagiarized? If you cant answer, them how do you know it was plagiarized
zacker150@reddit
Yes. I read all the original papers. Did you read the paper I cited?
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data by Gerstgrasser et al. points out that the theory of model collapse relies on the unrealistic assumptions that you delete old data.
To that end, in this work we study the effect of accumulating data on model collapse, rather than replacing data. Our data-accumulating setting is, in some sense, maximally pessimistic: it considers a hypothetical future where synthetic data are uncontrollably dumped on the internet to be vacuumed up for training the next iteration of generative models. Nevertheless, we find that model collapse is avoided when accumulating data.
To
Valmar33@reddit
That is not what I have been reading ~ the studies I have read involve feeding LLMs their own output, without deleting old data. Eventually, the models still collapse after multiple generations of feeding in LLM output, because of how LLMs fundamentally function. The weighted data eventually becomes more weighted towards the mean, because of how LLMs tend to select a variation of the most probable outputs based on the training data.
These papers you reference seem almost desperate to ignore the fundamental flaws in LLMs by pretending that the actual problem is something else.
Besides ~ if you don't delete old data, the LLM will begin to become "confused" over time, because as you feed more training data into the model, the weights change in more and more unexpected and strange ways, which is why they appear to "hallucinate". So you need to actually throw out the old data and start anew. You cannot "untrain" a model or "teach" it.
Tolopono@reddit
Been hearing about model collapse since 2023.
Meanwhile, llms have been training on ai generated data intentionally. Where do you think the reasoning traces come from
Valmar33@reddit
An appeal to popularity means absolutely nothing. Nor do appeals to authority.
The whole point is its slow, insidious nature ~ because it exposes a major weakness in the fundamental architecture of LLMs always eventually and inevitably ending up producing normalized results. That is, the most common elements get favoured more and more, with the fringe elements getting chosen less and less statistically.
Which LLMs are being trained on LLM-generated data intentionally? If that weren't a problem, they wouldn't need to keep sucking up more and more human-produced content at vaster rates.
Which "reasoning traces" are you talking about?
Tolopono@reddit
You dont seem to be particularly bright so you should ask Gemini to do the critical thinking for you https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%5B%221V3nF-ylRdRbZzXiZMqIk8aTE0Hq-5uVZ%22%5D,%22action%22:%22open%22,%22userId%22:%22112358100894503855820%22,%22resourceKeys%22:%7B%7D%7D&usp=sharing
Valmar33@reddit
If you let LLMs think for you, you will lose the capacity to think creatively and critically.
Ironic, that you're asking me to let something else think for me ~ when you seem unable to think past the LLM hype.
Tolopono@reddit
You never seemed to have that capacity to begin with
Valmar33@reddit
And now you're just slinging meaningless insults around without evidence.
Tolopono@reddit
Bro doesn’t even know what an appeal to popularity is and expects me to waste my time lmao
Valmar33@reddit
You wasted enough time responding to me on the internet.
LLMs are massive popular in the corporate space, and you just threw a bunch of random quotes from what you consider big-name programmers, as if to convince me that they know what they're talking about, as if that means there's a good reason that it's being used more and more.
Just let an LLM do the thinking for you. I have better things to do ~ like think.
Tolopono@reddit
If they use it, why wouldn’t other devs
Valmar33@reddit
That would be the appeal to authority and appeal to popularity ~ "these big-name individuals use it, other people use it and like it, so you should too!"
Classic logical errors ~ something being recommended because some perceived authority uses, or because something appears to be popular, is not an indicator of that something being actually good or worth using.
Tolopono@reddit
https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%5B%221V3nF-ylRdRbZzXiZMqIk8aTE0Hq-5uVZ%22%5D,%22action%22:%22open%22,%22userId%22:%22112358100894503855820%22,%22resourceKeys%22:%7B%7D%7D&usp=sharing
_TRN_@reddit
It’s kind of sad that you’re letting the LLM think for you here.
Tolopono@reddit
Its what op deserves for saying that was an appeal to popularity
_TRN_@reddit
I agree that "appeal to popularity" isn't quite right here. Some of the examples you have posted here are legitimate but the mistake you're making is in assuming that those examples are substantive enough to spell the death of programming. In reality, all it means is that LLMs are gradually getting better at unsophisticated programming tasks. This is not to say LLMs cannot do sophisticated things but that you should not treat them as infallible and you still need good judgment to make good use of them.
I wouldn't pay too much mind to the stuff Karpathy says. Yes, he's a talented AI researcher but he is not a software engineer. It's also hard for me to trust him at this point due to his unwillingness to denounce Elon and his part in marketing Tesla's auto-pilot as FSD when it clearly wasn't (and still isn't).
Tolopono@reddit
Its much better than what this subreddit will give credit for
Valmar33@reddit
Reads like you being unable to think beyond LLM hype, unable to see all of its flaws, astonished that anyone could not think that it's the best thing ever.
So, goodbye.
dbkblk@reddit
Well, I kind of agree, but as an experienced dev, I'm using it for some tasks. You just have to do small flee jumps and check the code. For small steps, it's good. However, if you hope to dev some large features with one prompt, you're going to be overloaded very soon. I would say it has its use, but companies oversell it 🐧
Valmar33@reddit
I find it questionable even for small steps ~ because it's less painful and bug-free just writing it yourself, when you know what you want. You learn more that way ~ how to avoid future bugs and build it as part of something more complex.
If you write it yourself, you have a much greater chance of remembering it, because you had to think about the process.
With LLMs ~ you're not thinking or learning.
dbkblk@reddit
I also disagree. There are often tasks that you know how to do it, but it's faster to ask the llm to do it instead of doing it. You learn new things when you're trying new things, not when it's the 20th you do it (and I'm not even talking about boilerplate, but once you did many projects).
Valmar33@reddit
If you don't keeping training a muscle, eventually it will atrophy. It becomes lazy and weak over time, to more you rely on a crutch. You will eventually forget how to do something without practice.
dbkblk@reddit
I agree! That's why I take notes of everything 🙂 Because forgetting how to do things is part of the job! There's just too much to remember so it's better the remember the whole frame and the logic to do it, not the actual code. I was working like this way before Ai become a thing.
Valmar33@reddit
That's why you practice, and don't rely on tools to automate ~ unless you've written the automation tool yourself to know what it actually needs to do, and will work properly, without uncertainty of bugginess.
Your code should be a good representation of your logic ~ else what exactly are you doing? If you let an LLM do it for you ~ it's not your logic or frame of thinking.
dbkblk@reddit
I think we kind of view things the same way, but opted for different stances.
At work, I never use any LLM, because it's forbidden, and I don't really need to. For other projects (and I have a lot), I use LLM to help me get faster on track, but most of the code is written by me anyway (I would say 75%).
Valmar33@reddit
And how often do you have to debug the code the LLM gives you? Do you actually understand and comprehend what the LLM is doing?
dbkblk@reddit
Well it's hard to state on this, but I check the code everytime anyway! It depends on the difficulty of the task. On hard one, you have to debug more obviously, because the LLM can go wild. Of course, most of the time I understand what's happening, and if I don't, I pause there and do some research. The thing is to never ever let some code you don't understand in your codebase. That's why vibe coding without knowledge or research is bound to fail!
zlex@reddit
I really disagree. For rote contained small tasks in coding, especially repetitive ones that follow the same refactoring pattern, I find LLMs are much faster and actually make less mistakes
DerelictMan@reddit
Agree. I definitely get the impression that many in this thread are solo devs. When working on a feature with a coworker, sometimes the coworker takes some rote task that is mostly boilerplate and handles it. When they do, I am thrilled that I didn't have to do it. Replace "coworker" with "Claude Code" and the statement stands.
FeepingCreature@reddit
This is not correct. Developers with an established codebase get no meaningful benefit from LLMs such as they were a year ago.
Valmar33@reddit
LLMs only function correctly if they have large amounts of training data ~ and you would want to train it on that established, well-understood codebase in order for the LLM to give you useful output in line with that codebase.
Ironically, at that point, you don't need an LLM, as you have a team understands the codebase well-enough.
New people should be trained by experienced developers on the codebase, so they can understand how and why it is written as it is.
An LLM will not help them learn ~ they will just become reliant on the LLM, and won't think about how to actually write the code.
Sparaucchio@reddit
I did not write my colleague's code, and debugging it has always been a pain in the ass. Weak point imho, unless you are a solo dev...
PurpleYoshiEgg@reddit
I can get on a call with my colleague and ask the reason they did what they did.
If I ask an LLM, it's just going to bullshit a reason or try to overcorrect itself.
Smallpaul@reddit
So the minute you leave the company your code becomes a “total nightmare” for the person who comes next? When your colleague is on vacation you consider their code a “total nightmare?”
Well written code should not be a “total nightmare” to debug, whether written by human or machine.
LeakyBanana@reddit
I think I'm starting to understand why some companies interview by just putting code in front of someone and say "Figure out what's wrong with it." Apparently the ability to do this is a huge problem for many in the industry and in this thread.
HommeMusical@reddit
Machines have no concept of "well-written code".
entertheclutch@reddit
5 years ago, machines had no concept of what a banana is either, today they probably still dont but there are certainly LLMs that can produce output which meets most peoples expectations for demonstrating understanding of a banana
Smallpaul@reddit
Flatly false.
Just try it. Write some idiomatic, clean code with good variable names.
Write some messy crap with confusing variable names.
Ask a modern LLM to compare them.
We can quibble about whether feature vectors or concepts but that’s irrelevant to the question at hand.
Your dismissal of the technology demonstrably relies on you misunderstanding its capability.
Before every push to a branch I always ask an LLM to critique the code whether I wrote it or whether it did. It often finds useful improvements, whether the code was written by a human or an AI.
Valmar33@reddit
Only if there is no-one left who has dealt with that person's code and understands how to review it. But, yes, that can happen for some companies, unfortunately.
LLMs are not known for writing "well-written code", lmao. Humans at least understand what they have written ~ because they form a mental model of it while writing it.
LLM-generated code will never produce such understanding ~ because you're not thinking about the code. You're just generating it, and then have to debug a possible nightmare you don't comprehend, because you didn't write it.
At least by writing it yourself, you can understand what you are doing, and what mistakes you might have made, when reflecting on your own code.
chjacobsen@reddit
"Reliable developers statistically get no meaningful benefit from LLMs \~ LLMs just slow experienced devs down as they have to spend more time debugging the code the LLM pumps out than if they just wrote it from scratch."
I think that's far too categorical. There's a space inbetween not using LLMs at all and full vibecoding with no human input.
Not all LLM use compromises the structure of the code. It's very possible to give scoped tasks to LLMs and save time simply due to not having to type everything out yourself.
xmBQWugdxjaA@reddit
A huge space. Like in my team we use Claude Code with the superpowers plugin to have loads of human input at every step of brainstorming -> implementation -> review - it's exhausting, but well worth it.
We then have lazygit with difftastic and delta to make it easier to review specific changes, before pushing.
Then on Github it can be reviewed and all the comments and context fed back to Claude.
It's definitely a huge speed-up - making some migrations tractable by two developers pairing for a couple of days, compared to weeks of painful work.
That said, it definitely excels better at translation work (e.g. when it can clearly see both APIs, etc.) - trying to do anything on the bleeding edge is painful as you have to constantly get it to search the web so it doesn't rely on outdated docs, etc.
Smallpaul@reddit
Curious why you need both difftastic and delta?
Thanks for the tool recommendations.
xmBQWugdxjaA@reddit
delta is just a pager for syntax highlighted diffs (which I prefer, although some people dislike the extra colours), but it also helps to have it configured for git itself so you can still use git diff --patience (with histogram set as the default algorithm) outside of lazygit in case you hit an edge case with difftastic or the histogram diff.
Orbidorpdorp@reddit
This is also where like 90% of professional employed devs are at too. Nothing gets committed before you yourself review the diff, and then the PR itself gets reviewed by both AI and humans.
imp0ppable@reddit
Fully agree, in any large codebase there's going to be a constant need for tiresome maintenance PRs, fixes, dependency updates etc. Letting an LLM do that stuff is actually useful, it's the equivalent of delegating to an intern. You still have to review it but you would have had to review the intern's work anyway.
omac4552@reddit
Code is easier to understand when you write it yourself compared to reading. So I'm not so sure the measurement of created code lines really is something that should be accepted as a win.
Maintenance is going to go through the roof for the people skilled to actually understand the output of these LLM's, and they are going to spend a long long time understanding and debugging code when something goes wrong.
Me myself will find other things to do than code reviewing LLM's, I'll leave that to others to do.
ffiarpg@reddit
I wasn't saying created code lines was the benefit, reduced lines of code required from a human is the win. Several others mentioned it requires more oversight on those lines and that's absolutely true. The question is whether it is a net gain and in many lines of work it certainly is.
Code is often read months or years later, often times not by the person who wrote it. By the time you would see a benefit from the understanding you gained writing it yourself, it has already faded.
AutoPanda1096@reddit
It's nuanced.
I've been coding for 30 years and these tools allow me to dip into other languages without having to go through the same pain I used to.
I'm not just saying "write me this app"
I'm approaching coding just the same as I always have done
What's the first thing I want my app to do? Open a file. "AI, teach me how to open a file with language x"
And then I read and understand that.
Obviously it's impossible to share my process in a two minute Reddit reply, I'm just trying to give a gist.
But with AI my ability to pick up new things and work on unfamiliar things has accelerated by orders of magnitude.
We now have a local LLM that can can point us to bits of code rather than hours of painful debugging. "This field is wrong, list out all the objects that can affect the field"
Like I say, it's hard to explain and I've argued this enough to know people go "but you're missing out on X and y"
I just don't buy it.
It's like teaching kids to hand crank arithmetic when calculators exist "but you have to learn the basics!"
It's a bigger debate than I'll ever take on via Reddit lol but check out professor Wolfram's views. We need to teach people how to use tools. Don't teach them to be the tools.
EveryQuantityEver@reddit
Calculators are deterministic. LLMs aren’t. They can and will make stuff up if it seems plausible
red75prime@reddit
Nondeterminism of LLMs' output is not intrinsic (they produce probability distribution of tokens, but sampling can be done deterministically). And it has nothing to do with hallucinations, which are statements that have high probability despite being wrong by some criteria.
omac4552@reddit
I know it's nuanced, I'm just saying I won't spend my life maintaining LLM code and review LLM code.
I also has programmed for 30 years, right now I'm implementing passkey login for a financial institution website and app, and when I tried to use LLM's it messed everything up and got it plainly wrong.
I normally use LLM cautionary for the boring stuff because I like to make my code clean, clear with intent, naming humans can understand and flows that are easy to follow. This is something I create in the process by doing it, because I don't know what to ask for when I begin.
axonxorz@reddit
My experience in existing codebases is pretty negative, often long spins for an 80% solution. Yes, that's partly my code's fault, python with a lot of missing type hints that could assist, but this is a legacy codebase started in 2017.
Where I have had the most benefit is exploration of approaches. I'll create a greenfield project and ask for as little as I can to get my idea out. It's a great way to see a "how" that would take hours researching through ad-hoc web searches.
But then I completely throw away the LLM code. It's never sufficiently structured for my project (yes, this is again my fault).
I'm working on a user-configurable workflow system in my application (very original lol). Version 1 is running, but version 2 needs a ton more features and the ability to suspend execution. I had absolutely no clue how to approach that, so I asked an LLM. Not a single line of that code ended up in my production app, but knowing the approach was all I needed to continue.
thereisnosub@reddit
Hahaha. Is that considered legacy? I literally work all the time with code that was written in the previous century.
omac4552@reddit
"Not a single line of that code ended up in my production app, but knowing the approach was all I needed to continue."
It's also my experience in general, most often you only need someone to point you in the right direction to get started
PotaToss@reddit
It's not 1:1 with a calculator, because LLMs are built to bullshit you, and when they do, you're being saved by your 30 years experience hand cranking it.
I think senior+ devs can use it reasonably. I think most of the problem is that you get bottlenecked by people with the judgement to screen their output, and if juniors and stuff are using it, it creates a huge traffic jam in orgs, just because nobody's really built top-heavy with seniors.
vlakreeh@reddit
I don't understand this, presumably most of the code you interact with day to day already isn't written by you but instead written by your coworkers. Unless you just don't review your coworker's PRs then I don't see how this is that much different, the current SOTA models don't really generate worse PRs (at what I've been working on recently) than juniors I've worked with in my career.
omac4552@reddit
As I said, code reviewing LLM code is something I will choose to not work with. Code review in general are boring and we don't do much about it in our team. The amount of LLM code that's going to be produced I leave to someone else to read. By all means this is a personal choice of what I want to do with my life, everyone else who feel different about can do whatever they want to do.
And before someone are losing their head because we don't do much code review.
We are a small team that delivers a huge amount of value, we are self organized and do not follow any methodology other than common sense and don't be stupid. We are working in finance and trading and probably do 5-20 deploys to production each day.
And yes, it works, it moves fast, feedback loop are lightning fast and bugs are fixed immediately.
Valmar33@reddit
Precisely ~ because it was written with your mental framework in mind.
With an LLM, you have no idea about the design decisions or how to mentally parse it. If it's a bug-ridden mess, you could be stuck for a very long time. Better to just write from scratch ~ at least you can understand your own bugs that way, and become a better programmer, as a result.
DownvoteALot@reddit
I don't know how you write code but we do pull requests and at least one team member has to approve before we can submit changes. That person has to understand the code fully and make sure others will understand it too, doesn't matter if written by LLM or not.
omac4552@reddit
So the person who code review are now responsible for understanding what the LLM created, I'll pass on that job.
Tolopono@reddit
Or just ask the llm to explain it
ourlastchancefortea@reddit
Suuuuuure
AutoPanda1096@reddit
Haha yeah the reality is that other people in your organisation have and always will check in shitty code.
Getting to grips with that is part of being a corporate professional.
All this means though is that point above is moot
Not having a code base that matches your personal architecture is always going to be a problem because you'll never write 100% of the code when you're hired into a team.
fripletister@reddit
Anyone care to provide context about why they downvoted this user, or...?
Seems like a pretty reasonable take. In a team-based environment everyone has different styles and approaches to software development, but team members might still have to understand each other's code at various points. Maintainers are responsible for it, since they're merging it, etc. This does in fact minimize the point made above. Where's the logical error?
EveryQuantityEver@reddit
Because, “Other things make mistakes too!” isn’t a reason to turn everything over to an LLM which doesn’t actually understand anything
fripletister@reddit
Nobody is "turning everything over to an LLM". That's the point. It's a tool. If it's useful, it's useful.
Maybe some of y'all are afraid for the future of your careers? Maybe if I was you I would be too. Denying reality isn't going to help you, long or short term. And this is coming from someone who never wanted any of this shit.
mpyne@reddit
So if that's not already happening (and you're right, it's not), how can we say LLMs are actually worse than what's happening now?
At least for software, for all we know what they're doing may be just as good, if not better.
Valmar33@reddit
With an LLM, that is more difficult the higher in complexity the code in question becomes ~ only by writing it bit by bit yourself can you actually understand, and perhaps even explain it. With LLMs, good luck explaining the reasoning...
Philluminati@reddit
Even if you ChatGPT to explain some code it will write a 2000 word essay instead of just giving you a 6 box domain model diagram with a few relationships and a 5 box architecture diagram with a few in and out arrows, which is how most devs explain a system to a new person.
PurpleYoshiEgg@reddit
Plus, if you write it yourself, you can see if the architecture you had in mind was a good idea. It reveals warts.
With LLMs, I will not know if the issues it encounters are because it's writing buggy code or if they are exacerbated by poor architectural decisions. It makes everything more of a black box if it's relied upon.
PurpleYoshiEgg@reddit
How do you enforce that standard? Do they merely affirm they understand it fully by approving the pull request? Or do they write up a technical analysis on the code being merged that others can review?
mosaic_hops@reddit
LLM generated code often contains all kinds of subtle bugs that reviewers don’t typically anticipate. So it takes a lot longer to review and validate and creates these long, drawn out PRs.
Smallpaul@reddit
Human generated code often contains all kinds of subtle bugs that reviewers dont typically anticipate.
Valmar33@reddit
Humans write or type code ~ they do not "generate" it.
Human-written code may have subtle bugs ~ but at least the writer will be able to understand it, having written it proper.
People reviewing someone's actual-written code may also have an easier time parsing it if they've seen code by that person before, as they can begin to get an idea of how they think through how they write their code.
9uYx3QemUHKy@reddit
https://www.merriam-webster.com/thesaurus/generate
Let's not be silly
HommeMusical@reddit
Human-created bugs are much easier and more predictable to debug that LLM-based bugs, because we too are human.
Smallpaul@reddit
Not in my experience.
archialone@reddit
Why would I spend my time trying to decipher some one else code that was gnenerated by chatGPT?
I'd rather reject it immediately. And let you figure it out.
Apterygiformes@reddit
How do you know they understood it and didn't just approve it to get that slop out of their sight?
lord_braleigh@reddit
Knowing when to hold the line and where to let your coworkers run wild is the job at the staff and principal level.
flirp_cannon@reddit
If someone started submitting LLM generated PRs, not only will I be able to easily tell, but I’d fire their ass for wasting my time and their time.
xienze@reddit
100 file, 5K+ LOC pull request
LGTM
Tolopono@reddit
You can try asking it
AutoPanda1096@reddit
This argument breaks down when you remember multiple people contribute to any code base.
Any professional will have to work on code that doesn't match their "mental framework"
Valmar33@reddit
The difference between an LLM and an actual person is that actual people reviewing the code of other people can learn to understand their coding styles and thought processes behind the code. Actual people have patterns and models for coding implicitly built into the code ~ the variable names, the structure, even when they are following the coding guidelines, they will put their own flavour into it.
LLMs so often do not result in any significant speed-ups over time ~ these algorithms often result in more time wasted debugging the weird and strange problems created by them.
You are better often thinking about the architecture of each section, and then building it yourself, each and every step, as you are basically solidifying the model and concept of it in your mind as you type it.
LLMs are not "hired help" ~ it is not a person. It is a mindless algorithm.
Proper-Ape@reddit
Any professional will be able to tell you that for a large enough codebase they usually only have a good mental model of the code they've been actively working on, a weaker mental model for everything interfacing with their code, and almost no mental model in parts that are further away from their area.
Also the people that joined a project later tend to have weaker mental models since they couldn't contribute the same amount as the initial developers.
This often leads to the newest developers at some point asking to do large refactorings. Which usually doesn't lead to objectively better code, but code that fits their mental model better. Which may in the long run be better if the original developers left the project already.
At least in that situation a rewrite of sizable portions of the codebase becomes much more likely, and has the benefit that you have people that intimately understand it again.
Smallpaul@reddit
If you have “no idea about the design decisions” of the code you delegated to the LLM then obviously you are using the LLM wrong and that’s entirely on you. It’s not a free agent. You tell it what to do.
Valmar33@reddit
Obviously, you prompt the LLM with what you want ~ but you have no idea how the LLM will respond or why. LLMs aren't magic ~ they don't just automagically know what to do. There's no intelligence or reasoning skills.
Smallpaul@reddit
No, they aren’t magic. But they are obedient. They do roughly what you tell them to do. It’s flatly false that you have “no idea how the LLM will respond.” I have literally never asked an LLM to produce Python code and got C++ as a result. So obviously the result is somewhat predictable. The more careful your prompting, the more predictable the output will be. You are the one treating them as magic instead of instruction-trained text completes trained to follow instructions.
Valmar33@reddit
That is not how LLMs function ~ LLMs are very fancy prediction algorithms trained on other people's written codebases.
Prompt "engineering" is just worse in every way than writing it yourself, and learning along the way.
You are treating like LLMs like magic ~ you are personifying an algorithm ~ if you think that LLMs are "obedient" or "do what you tell them to do".
LLMs are absolutely worthless on anything the model hasn't been very extensively trained on.
Smallpaul@reddit
I absolutely agree with you that they are worthless on anything that they have not been trained on.
And they have been very extensively trained to follow instructions and write code.
Valmar33@reddit
Again, you are anthropomorphizing an algorithm ~ these algorithms do not "follow instructions" or "write code". These are algorithms that take inputs, and give you outputs based on the training data, with a good flavouring of randomness.
Your problem is that you are personifying these algorithms ~ you are confusing appearances with reality. In reality, there is just an algorithm ~ there is no decision-making or intelligence or creativity or anything else.
LLMs consistently create more problems that have to debugged than actual working code ~ at that point, you're better off just writing code that is abstracted properly as necessary.
6890@reddit
Dingdingding
LLMs work in the same way StackOverflow worked. If you've got a problem that's not novel or new, you can probably find an answer in its wide breadth of training data.
But as soon as you've got a problem that goes beyond something done a hundred times by others it struggles and starts to hallucinate. That's when you have to be extra careful that all the edge cases and pitfalls are covered. Its when you start wasting time holding its hand and lose any of that promised efficiency as you delicately tweeze each function, each decision for what it did and what it didn't do.
I'm half convinced that the LLM evangelicals fall into two categories:
1) the upper level suits who were sold a lie that these are true AI and will replace labour, lowering costs and increasing efficiency
2) newbies-to-intermediates who haven't really had to create something new and novel. Because if all you've ever done is rewrite a bunch of boilerplate or take something and alter it in minute ways an LLM can look like magic
Maybe there's the 3rd category for the amateur. Who asked an LLM to do something and it delivered. But they never put it out to the public or rigorously tested it. Sure, you can create a little checklist app, and i'm sure it works if you only do the essential stuff it was meant to do. But it would crater under public use where people try to break things, or worse, it is impossible build upon and remains essentially niche or useless.
syklemil@reddit
This also kills off the important signal that "no results" is. Searching for something very specific and getting no results is an indicator that it's a wild goose chase. Getting a ream of hallucinated ways of doing something that is actually unsupported is an extremely frustrating use of anyone's time.
stimulatedthought@reddit
Preach!
happyscrappy@reddit
I always think of back when I used to race RC cars.
RC cars were bought as a kit. You built it and added some electrical parts and ran it. But a lot of the people on the outside didn't see why you would want to spend all that time building it before you could run it. Surely a pre-built car was a better idea. Think of the gift opportunities among other things!
So two new RC aficionados go out and drive their buggies around. And because they are so fast and the nature of the fun within a few days both of them break their buggies. For the one who built his buggy, it's no big deal. You already built the rear suspension the first time, just go get replacement parts and follow the same steps you did before.
But for the one who bought it prebuilt everything under there is new to him. At the very least you have to go back and learn all that stuff you avoided learning in the first place by buying it pre-built.
Your software is going to need to be altered. To fix it or change what it does. It's going to be a more difficult and intimidating process to do that if you don't know how it was put together in the first place.
To a large extent having an LLM write the code is false economy. You'll give back most of those savings when you continue to use the code. The question to me is really how much do you give back? 50%? 95%? 130%? I don't know there's a way to know ahead of time.
Woaz@reddit
Well if youre not “vibe coding” files or directories at a time, focus on generating a single function or code block, and then making sure it makes sense, its not too hard to understand and can definitely save some time just typing it out if nothing else.
All that to say its not perfect and comes with drawbacks, but its probably one of the more reasonable use cases (along other draft-and-verify applications, like writing a letter/email). What really boggles my mind is basically taking this unreliable source of information and using it in situations without verification, like live for customer service, product descriptions, or straight up “vibe coding” without understanding it.
omac4552@reddit
I do use LLM's but I find them very limiting in understanding my code space and create what I want. But yeah, mappings, casting from bytearrays to base64/string, memorystreams etc. which I never remember the syntax for they are fine, even if they miss my intent in that space sometimes.
Somebody is now going to tell me I'm using them wrong, because that's always the case.....
doiveo@reddit
So give your Ai a style guide and rigours rules around structure and architecture. Templates and negatives are the key to getting code you would use. Every project needs a decision file where anything you or the Ai chooses gets documented.
In the end, the code becomes disposable - it's the context that must be engineered and maintained.
Helluiin@reddit
not just coding but everything. theres a reason schools make you write and work out so much on your own, because its proven to improve your memory of it.
MSgtGunny@reddit
Reading and debugging code you didn’t write causes burnout faster than writing your own code.
LeakyBanana@reddit
I'll never understand this argument. Are you all solo devs or something? You've never worked on a team codebase? On a codebase with multiple teams contributing to it?
Y'all are only ever debugging your own code? Do you just throw your hands up and git blame any time a stack trace falls into someone else's domain? Maybe understanding and debugging others' code is a skill you need to spend some time developing. Then maybe working with an LLM won't seem so scary.
jug6ernaut@reddit
Writing code is easy, designing code is hard. The vast majority of development time isn’t writing code, it’s ensuring what is being written makes sense from a business, maintenance and reliability perspective.
With blindly using LLMs you throw away these concerns, so you can speeding the easy part. The larger of a team or project you are in and on the harder all of these thing become.
LLMs make these problems harder, not easier. Because now you know nothing, and in turn can maintain nothing. Oh and hopefully it didn’t just generate shit.
The standards we have for design, maintenance and reliability should not change bc LLMs can make the easiest part of development easier, if anything they should make them more stringent bc the barrier for is now lower. That doesn’t mean we shouldn’t use LLMs, they are an amazing tool. But just as you shouldn’t blindly copy code from the internet before LLMs, you shouldn’t blindly copy code from an LLM now.
LeakyBanana@reddit
Rather presumptive of you. You won't find me advocating for "blindly" copying LLM code. I'm talking about the opposite, actually. Reading and understanding code you haven't written is a core skill that many people here need to work on developing if they're really that concerned about their ability to use an LLM for code generation.
Personally, I spend a lot of time reading and iterating on my own code to improve its quality. I'm a tech lead and I spend a lot of time reading others' code and suggesting improvements for the same reasons. And it's really no more difficult for me to ask an LLM to refactor towards and improvement I had in mind than it is to ask someone on my team to do so on their code. If you want to get anywhere in your career, it's a skill you need to work on. Then this won't seem like such an insurmountable hurdle for LLM usage.
omac4552@reddit
You're free to spend your life on whatever job you want, I fortunately also can decide what I want and not want to spend my life on. Code reviewing generated code and own it I’ll pass on but, there is probably going to be plenty of jobs for those who seek those opportunities.
vulgrin@reddit
“they are going to spend a long long time understanding and debugging code when something goes wrong.”
Seriously, no they won’t. Because you use the same tools to debug and explain the code that you use to write it. I can with an LLM and my decades of experience pull up a completely foreign code base and understand what’s going on and where the critical code is quickly. Searching and doing debugging by hand and with the LLM is trivial and the same as it ever was. Then writing the prompts to fix code that’s already written is easier (in most cases, UI notwithstanding) than the initial build.
If you are reviewing changes every time an LLM makes it, you’ll understand the code just fine and catch the problems. In my experience the more mature the project is, the less issues I have and the more I can trust the project because there’s enough examples for the agent to follow.
It’s really strange to me that we programmers have been given power tools and everyone would rather sand by hand. Like woodworking, hand craftsmanship is good for some projects but when I’m just building a shed, I just want it done.
omac4552@reddit
You're free to spend your life on whatever job you want, I fortunately also can decide what I want and not want to spend my life on. Code reviewing generated code and own it I’ll pass on but, there is probably going to be plenty of jobs for those who seek those opportunities.
Uristqwerty@reddit
You know the saying "If I had more time, I would have written a shorter letter"? AIs make generating new code so easy that I'd expect the size of the project to expand until it bogs down new development more than the AI allegedly sped things up.
Every line written is a line future programmers must read and understand. If they don't understand, there's a risk that when adding a new feature, they'll carve out a fresh file and re-implement whatever logic and helpers they need, duplicating logic. Or worse, a near-duplicate with different bugs than each of the other 5 copies that have accumulated.
archialone@reddit
Writing large amounts of code was never the issue, understanding the system and debugging, designing solutions that fits to the problem were the issue.
Having LLM spit out vat amount of text is not helpful.
ptoki@reddit
there is a point where
"in php write me a loop which iterates over an array of strings and returns concatenated string consisting only rows matching pattern *.exe"
And
"$result = '';
foreach ($files as $file) { if (fnmatch('*.exe', $file)) { $result .= $file; } }
echo $result;"
are equal in complexity or the prompt is much more tedious to compose than the code itself.
I still dont see revolution and chatgpt is with us for like 3+ years...
SmokeyDBear@reddit
This is 100% true but it assumes an answer to the question “Is not having more code the thing that’s keeping us from making progress?” (or, more importantly, “is not having more if the type of code that AI can write the thing that’s keeping us from making progress?”). Maybe the answer is “yes” but it’s probably worth making sure.
chjacobsen@reddit
We're also very early in learning how to actually apply LLMs to coding.
LLMs themselves are not reliable, but we can do a lot to constrain them and make failure cases rarer and easier to catch.
Topics such as which programming paradigms we choose, our testing tools, static analysis setups, which programming languages we choose, how we manage context to avoid tunnel vision - all of those make a huge difference to the reliability of LLMs, and we've barely even begun to explore those things.
The more I dig into it, the less concerned I am that programming will be done by vibecoding marketing managers, because I actually think the emergence of LLMs makes the job harder in some ways. Creating the environment in which a non-deterministic AI model can be run reliably takes a lot of effort, but the rewards can be software that is both quick to write AND better than what we used to have. In that space, the market for slop isn't looking great.
zoko_cx@reddit
There are two challenges which user (programmer) needs to overcome to let say LLMs could increase his output.
First as you mention he needs to know how to use LLMs and for what kind od task are useful and for what not or lot less. Second is how agentic coding works and how to better setup it with LLMs, controlling context etc.
Second by most important thing is you need to know is correct code by design and this is where knowledge and experience come to play. If LLM output some code which you never saw before you need to understand it, know if it good solution of problem. So maybe we should less focus on mastering writing the best code but more to unit/integration testing, refactoring, security and overall system design and architecture.
SwoopsFromAbove@reddit (OP)
Absolutely, and it’s very cool to be able to do that! The problem is that societal assumptions are that the computer is always right - challenging computer output doesn’t come naturally to us, and we don’t have the systems in place to do so effectively.
This is encouraged by the LLM vendors, who have a very strong financial interest in framing their tools as all-powerful and super-intelligent. They rely on the strong psychological priors we have to trust computer-generated answers to oversell their products’ capabilities.
zoko_cx@reddit
Down vote fore 1s paragraph but I would up vote for 2nd.
Computers are stupid machines, they are in category of calculator but much complicated with more capabilities.
But still they output is base on human input. People who think that computers are smart are just very naive and probably they won't manage to use computer for simple task without ragging.
MrTroll420@reddit
Are you from eastern Europe? I agree
backelie@reddit
Citation needed
aaron_dresden@reddit
I don’t know anyone who trusts that the computer is always right, even before LLM’s. Computers have always gotten into inconsistent states at times, and there are those who just inherently don’t like technology.
This is a weird way to frame things.
Seref15@reddit
Yeah a skilled person + an LLM together is just undeniably efficient. It's trying to get rid of the skilled person where things go sideways.
longshot@reddit
While it isn't reliable, I would say pure human effort is also unreliable in many ways.
efvie@reddit
Say it with me: code is bad, you should have as little code as possible. More code is bad.
(This aside from 95% wildly overstating even the unit-level correctness let alone modules or entire systems.)
Helluiin@reddit
95% is probably wrong even for a single statement depending on the language or library in question
_JustCallMeBen_@reddit
God IgM the 5% that is wrong requires you to read and understand 100% of the code.
At which point you have to ask yourself how much time you saved versus writing 100% of the code.
stimulatedthought@reddit
Disagree with the idea that humans aren’t reliable. SOME humans are not reliable but since we are the only truly “thinking” entity capable of programming in the known universe—the best of us set the standard for reliable in that regard. The expectation of those who demand things for perfection is the problem and comparing a confidence trick with true problem solving is where this gets complicated.
editor_of_the_beast@reddit
Right., it’s in the name: artificial intelligence. It’s emulating human intelligence, which is completely fallible. And we seem to have a functioning society even with that.
ub3rh4x0rz@reddit
The last sentence is debatable
HommeMusical@reddit
I would not work with a developer who had a 5% error rate.
They will produce a larger volume of code, for sure.
BoringEntropist@reddit
Most code out there in production isn't maintained already. And you want to add even more code? We already know LOC is a horrible metric for decades, as it leads to bloat, security vulnerabilities and economic inefficiencies.
Crafty_Independence@reddit
That's not why many companies are using it though. A good percentage are using it because the C-suite thinks it will allow them to replace human workers and/or the shareholders are clamoring for AI usage.
Very little of the hype is being driven by data.
fractalife@reddit
Studies have so far shown this not to be the case. It's about the same or worse. Developers have always made tools to automate tedious repetitive code, or if possible template in a way that it's not necessary to do. That's kindof the point, after all.
That's where LLMs excel, so they're filling a niche that has kindof already been filled. When it comes to novel approaches to particularly interesting problems, the LLMs are just going to guess, because they aren't actually curious and don't "want" to solve problems. They're just programs and marices at the end of the day.
Sisaroth@reddit
Exactly, I don't understand why anti-ai redditors are so hung up about LLMs not being correct 100% of the time. It's still a very useful tool even if you should (almost) never trust it blindly.
For example: I hate CSS/html styling but getting it wrong should never be a security risk. This is the one exception where I will use LLM generated code without reading it first because there is no security risk in doing so.
Another example: You are stuck with some problem and the solution to it is spread out within different pages of documentation. A human could easily spend 4-8 hours digging through the documentation to find the solution, an LLM can often do it in one shot. You can ask the LLM about it's sources and then you can double check that it actually came up with the correct answer and not just a hallucination. You just saved a day of work (i have seen this scenario happen multiple times at work, both with myself and colleagues).
Valmar33@reddit
LLMs are only vaguely good at very basic code ~ but complex projects are a nightmare, because you will be unable to reason about all of the moving parts. At least if you write it yourself, you will develop a mental model of how it flows in your mind ~ because it was written in accordance to how you think as a programmer.
ChemicalRascal@reddit
Because I write software that needs to be correct 100% of the time, and having to review an LLM's output is both not quicker than writing it myself and not as safe.
Getting styling wrong is an enormous usability risk. If you only care about security risks, I'm sorry, but you're an idiot. Now, sure, never put security under usability, but you have capacity in your brain to care about multiple things at once.
Just spend a weekend on YouTube and learn how CSS works. That's what normal devs do, for crying out loud. Unless you're literally on death's door you have tonnes of time to learn this basic shit.
logicality77@reddit
The cognitive dissonance comes from people assuming that, in order to produce the things LLMs are able to generate, thought and reasoning must be involved. Most people don’t appreciate that there are patterns in our speech, art, and definitely in our code. If it has a pattern of any sort, an LLM can be trained to mimic it. It doesn’t help that people use anthropomorphizing words like AI and even hallucinations to describe LLMs. Yes, machine learning is involved, but when laypeople think “AI” most people think of some kind of AGI, which is light years ahead of any technology we have today.
As far as companies relying on it, it’s just lip service to secure investment interest and capital. It’s the hot new technology, and most investors don’t have the savvy to know the limitations of the technology. It’s not much different than crypto and NFTs just a few years ago. Many companies were talking about their plans to incorporate NFTs in to their business somehow, but most of those plans are long forgotten today.
personman@reddit
i agree with you completely, but where did you come up with 400 years?
sickhippie@reddit
It's literally the title of the linked article, and it references the invention of the mechanical calculator in 1623.
personman@reddit
oh wow the fact that there was text in the post made me completely miss that there was a link, thanks
Kok_Nikol@reddit
It's a new-ish trend, to old school reddit users it looks like a self post, it took me a while to get used to it.
peligroso@reddit
Old school redditors remember the days when people were mocked for self-submitting thought-pieces from their own personal blogs.
Kok_Nikol@reddit
Eh yea, I mean, it's still frowned upon, but there's just too many people now to keep that in check.
It's too late to fix - https://en.wikipedia.org/wiki/Eternal_September
_illogical_@reddit
I know that Reddit, at least in the past, had kinda the inverse of this. There would be a huge rise of low quality posts when schools were out, like during the Summer, then drop when kids went back to school.
VeganBigMac@reddit
That's a similar, but slightly different phenomenon. Eternal September refers more the permanent degradation to community norms as the community grows bigger.
nullvoid8@reddit
It's literally the same thing. If "Eternal September" were to be named by Reddit (or at least the above Redditor), it would have been called the Eternal Summer. Both refer to a previously cyclical influx of new newb-ish users becoming a permanent state of affairs.
VeganBigMac@reddit
No it doesn't. Eternal September refers to a non-cyclical permanent increase, and the "Summer Effect" refers to a cyclical non-permanent effect.
aelfric5578@reddit
That's a new term for me. I like it.
kramulous@reddit
It is also nice, now, to go outside our standard set of sites and visit something new.
bdmiz@reddit
You’re absolutely right to ask. The “400 years” comes directly from the title of the linked article itself, which points back to the invention of the mechanical calculator in 1623—roughly four centuries ago. That’s the historical reference being used, not an arbitrary estimate.
The_Artist_Who_Mines@reddit
diabolical
badmartialarts@reddit
I thought they might be referencing the Mechanical Turk.
desmotron@reddit
From the LLM he used to make sense of the other LLM’s
LegendEater@reddit
Smallpaul@reddit
The article mocks OpenAI for being slow to release GPT-3 because OpenAI was concerned about it being abused. The article claims that OpenAI was lying because LLMs are safe and not harmful at all. It also links to the GPT-3 announcement where OpenAI said that they were reluctant to release it.
Why were they reluctant?
“We can also imagine the application of these models for malicious purposes, including the following (or other applications we can’t yet anticipate):
Generate misleading news articles
Impersonate others online
Automate the production of abusive or faked content to post on social media
Automate the production of spam/phishing content “
Good thing those fears were so overblown! Turns out those liars at OpenAI foresaw a world filled with blog spam and link spam and comment spam but good thing none of that ever happened!
qubedView@reddit
And how dare they make an attempt to take responsibility for a new technology they created which they don’t yet understand!
txmail@reddit
Did they though? Seems like this stems from tech created in the late 70's and 80's during the first AI bubble. We have done this before, we are just repeating it again but with more hardware and the bubble is much bigger this time. Look at the history of the Symbolics Corporation for a geeky history lesson.
If we got anything out of this bubble it is better compression techniques for raw text... though it is completely destructive and irreversible and actually is not compression at all (tokenization)...
Actually I am not sure what this AI bubble has given us that we did not have from the 80's. Just the same shit done faster. Full text search is still better than a LLM when you need to cite sources / find the original content or want actually relevant results not crawled a year ago.
Really the coolest shit is being done with machine learning, especially the computer vision side that lets us spot cancer in images or identify birds or bird songs. Even google knows this, and it is why you have to click the blocks with a motorcycle on pages to identify your not a robot, when really your helping tag objects for their computer vision models.
SocksOnHands@reddit
The paper "Attention Is All You Need", which makes modern LLMs possible, was published in 2017. AI, as a field, had existed for a long time, but that doesn't mean the techniques we use now had existed that long.
paradoxxxicall@reddit
We had the core tech down for a long time. That paper introduced a relatively small tweak that finally got us across the line into useful language models.
I’m not downplaying the impact of course, just pointing out that the meat of this tech is very much not new.
Smallpaul@reddit
I could list ten inventions from the last decade but the most important is the transformer. No they did not have it in the 1970s and 1980s.
They probably would have invented in the 1980s if they had modern hardware but they didn’t and they didn’t. They might have invented Linux in the year 1200 if they had computers. But they didn’t and they didn’t.
Xirious@reddit
That sure lasted a long time.
splork-chop@reddit
In the context of the article the author is correct. The AI money bros and technologists have been rabidly saying all sorts of inflammatory nonsense about how they're 'scared' of AI and how dangerous it will become. This fits into the main point of the article in that these people are being intentionally disingenuous to stoke fear so that people pay attention and get sucked into the scam. If the AI fearmongers just said "well of course you might get some extra email spam or fake social media posts" no one would pay attention. OpenAI and others are clearly taking advantage of this climate of fear by suggesting they might have to delay or gimp their software because "oh no what might happen." Either that or the're bullshitting to justify release delays.
drekmonger@reddit
Those people have been screaming about that for decades. The concern far, far pre-dates big money rolling in.
AlSweigart@reddit
https://en.wikipedia.org/wiki/AI_winter
Smallpaul@reddit
Can you please verbalize what you are trying to convey with your link?
AlSweigart@reddit
https://www.reddit.com/r/programming/comments/1qci8z5/llms_are_a_400yearlong_confidence_trick/nzmdahj/
Smallpaul@reddit
I have a lot of respect for your work historically Al, (that’s an A-L as in Albert).
If you still consider yourself and educator and plan to create new content for students then I would encourage you to try and think in a more nuanced way about the world that your “students” are inevitably going to need to live in.
If you are done educating then sure, go full ostrich mode. Put your head in the sand and pretend that AI winter is going to protect developers for the very rapid change that the industry is undergoing.
But if you aspire to educate people about how to build software in this new world then that won’t be very effective.
drekmonger@reddit
https://en.wikipedia.org/wiki/AI_boom
AlSweigart@reddit
lol, the fact that that Wikipedia page was only made in 2023 tells you what you need to know. I clicked on five of the reference links at random, and none of them mentioned "boom" or "spring". Stop trying to make fetch happen.
splork-chop@reddit
Technologist-philosophers have interesting opinions on the dangers of AGI - a concept far removed from the current discussions around commercial LLMs and related tech. People conflating these ideas either lack fundamental understanding of the technology or are intentionally misrepresenting it for personal gain.
drekmonger@reddit
If you had shown an LLM and its assorted tooling to a researcher in the 1960s, they might have called it AGI.
Regardless, LLMs are far further along the track towards complete generalization than what we had before. How we deal with LLMs is a taste, a preview of what will happen when we have "AGI".
In scare quotes, because I suspect when we do have a better candidate for the moniker, people will still be saying that AGI is decades/centuries away.
splork-chop@reddit
Sure but the notion of what an AGI might be has changed radically. An LLM can pass a Turing test but doesn't fit into contemporary neuroscience theory on consciousness and mind, which didnt exist back then.
Not at all, in fact LLMs are completely tangential and divergent from contemporary research on AGI.
Smallpaul@reddit
You, I guess are a mind reader and can always know when people are telling the truth versus lying. I’m not, a mind reader and I admit to strong uncertainty. But we have a lot of evidence that they could be sincere.
We have ample evidence that many people were sounding the alarm on these risks going back long before these businesses even existed and sometimes it was the same people.
Recall that Bostrum, who is not invested in any of these companies had called out the risks in the book Superintelligence.
Eliezer Yudkowsky has dedicated his whole life to this and he also is not paid by any of these companies and did so before most of them existed.
Also, Hinton, who has the Nobel prize and resigned from Google specifically so he could speak freely says these same things about the risks of AI. He does so on almost a weekly basis.
We also have their internal emails from the early days when they expressed the same fears. Their INTERNAL, PRIVATE emails said:
“The goal of OpenAI is to make the future good and to avoid an AGI dictatorship,” Altman wrote. “You are concerned that Demis [Hassabis, the founder of Google’s DeepMind AI lab] could create an AGI dictatorship. So [are] we. So it is a bad idea to create a structure where you could become a dictator if you chose to, especially given that we can create some other structure that avoids this possibility.”
Recall as well that the organization was founded as a non-profit. This was a recruiting tool because they believed that many top researchers believed that AI was dangerous and could be better managed by a non-profit. That’s how they recruited Ilya.
I am baffled why people think that Ilya could not possibly hold the same views as his one-time mentor Geoff Hinton. Or that e.g. Dario who was involved in all of the same circles could not feel the same way.
splork-chop@reddit
The technologists you've cited certainly have some interesting philosophical opinions on the threats of AGI. The current colloquial/societal discussions around "AI" is so far removed from AGI that it not be brought into the discussion. All of the Datacenters and GPUs and LLMs or RNNs cannot produce an AGI any more than 1 million monkeys with keyboards can produce the works of Shakespeare. The fact that Altman or anyone can conflate what they're doing with AGI means that they either fundamentally misunderstand the technology or they're being intentionally disingenuous.
drekmonger@reddit
You stuck "RNN" in there, like you know what you're talking about. But really, it's a red flag that your education is either seriously outdated or you just decided randomly that recurrent neural networks sounded "Star Trek" enough to buy you some cred, without understanding what they are...
...or knowing that (almost) nobody trains RNNs anymore, and haven't for a long time now.
Smallpaul@reddit
I find it quite strange that with the pace of change of technology in general and the way that it happens in unpredictable fits and starts that you think that we should not discuss a change as big as AGI until we can be confident that if is around the corner.
How would we be confident? How many algorithms are we missing between here and there? How long does it take to discover them? How do you know that they are not already in the literature but not just applied at scale yet?
The idea that we shouldn’t discuss these risks until we know exactly the date on which they will arrive seems deeply flawed to me.
AlSweigart@reddit
They want people talking about how AI will bring prosperity and profits. They also want people talking about how AI could cause a robot revolution against humanity.
They don't want people talking about how AI is boring and doesn't live up to the hype.
IAmRoot@reddit
It's not AI itself that I fear but what powerful people might do with it. If we actually get to the point of true AGI one day, there's nothing stopping these CEOs from creating robot armies to kill the 99% of the population who lost their jobs so they can have all of Earth's resources for themselves. We don't exist in their futuristic "utopias." AI doesn't have the motivation to slaughter 99% of humanity. Billionaires do if they can replace us all with AGI.
Putrid_Giggles@reddit
It doesn't have to be perfect at all. It just has to be "good enough". Whether or not it meets that standard yet is still up for debate.
RyanCargan@reddit
I think it's a bigger attention-economy / marketing thing.
There's a problem where some people don't seem to realize that, to paraphrase a certain person:
People already have a kind of subconscious assumption that risk/danger = opportunity/utility.
People who think they're opposing 'corpos' just act as free marketing for them most of the time.
The narrative gravity thing happens even outside of tech. Ever seen politicians court controversy for free press? Because reaching their target niche is what matters, even if the press appears to paint them negatively for most people. If they reach who they need to, the rest of the world can pound sand. Happens with some cult stocks too.
There was a recent vid, that alleged with some receipts, that a certain popular infotainment channel (that is maybe a little overly concerned about the threat of AI) got soft-conned into acting as free marketing/hypemen for the AI industry. Bootleggers & baptists type stuff if true.
Predictions without expiry dates are meaningless and unfalsifiable.
Vague open-ended threats create narrative gravity that can't be ignored if you believe them (and attract attention and funding).
They also divert attention from more concrete immediate threats, while also making the tech a scapegoat for individual bad actors.
gmeluski@reddit
It's my understanding that the people from Anthropic were more safety minded and that's why they split from OpenAI. If I'm getting the dates right, GPT-3 was released in 2020 and anthropic started up in 2021. Based on this, it's completely reasonable that the safety camp had influence over the release process and then when they left OpenAI had way less reason to give a shit about any of that.
Smallpaul@reddit
According to the blogger and many of the skeptics on Reddit, none of them were ever safety minded and it was all just a form of marketing.
EveryQuantityEver@reddit
Considering the stuff they were “concerned” about had no chance of ever happening, and they didn’t give a shit about the actual risks and dangers of this technology, yeah
Smallpaul@reddit
Please read the top of this thread. The things they predicted all did absolutely happen and a lot more.
csman11@reddit
Both views here are true, it’s not so black and white. There’s definitely some harms and the ones you called out are the most realistic ones, and they can all be summed up as “abuse of LLMs to spread misinformation”. I don’t think anyone should disregard just how harmful this is to our already broken and polarized societies.
But these AI labs and other companies in the AI bubble have also been overstating capabilities of LLMs to drive attention to the space. Framing those capabilities as “disruptive and dangerous” in the ways the article’s author is getting at, is overblown. These dangers attract the attention of the general public, which in turn attracts the attention of policymakers, which then turns into the AI industry capturing state regulators because they’ve convinced us “we need to move fast to make sure the existential worst cases are avoided”. The big one is obviously financial/securities regulation avoidance. They can extract tons of wealth from both institutional and retail investors by creating attractive signals in the stock market with their revenue cycles. In an ideal world they wouldn’t be allowed to do that, but for some reason the policymakers have bought into the idea that the AI industry is important to national security instead of seeing them for the rent seekers they’re trying to be.
ii-___-ii@reddit
And yet they still released it...
Smallpaul@reddit
What would you have done? Especially with the knowledge that competitors are working on the same thing?
ii-___-ii@reddit
OpenAI was founded precisely because competitors were working on the same thing. If they were actually concerned about negative externalities, they would not have released a general chatbot the way they did. The point here is that it was a disingenuous marketing ploy to begin with, not that someone forced their hand.
What would I have done? I would have actually made OpenAI be a non-profit focused on how AI is developed and rolled out, and stick to their alleged initial core values. I also would not have them put all their efforts into scaling up a LLM with no specific use case. Prior to GPT, models were typically trained for specific use cases, making them much more efficient, and much easier to test and deploy under a limited scope.
TheESportsGuy@reddit
As if the internet hasn't been headed that direction for at least the last 20 years. Maybe AI accelerated it a bit...but not that I could tell. Mainstream internet content has been trash and/or marketing since the AOL days. And the skill required to parse it hasn't changed with the advent of AI. Google search was a temporary hack for a while, but it also fell off as a viable way to find value in internet content long before GPT-3 was released.
Everything about AI is just the usual financial hype cycle, 20-100 years early.
Kusibu@reddit
There were garbage websites, but it used to be at least a little difficult to completely falsify an entire article on a breaking topic like five minutes after it happened, or generate a new fake website to match Google's SEO changes each time they happen.
bduddy@reddit
I mean they obviously don't actually care about any of that. We should not be taking what they say at face value.
Smallpaul@reddit
Okay, so if they didn’t care about any of that then why wouldn’t they our GPT-3 out into the world and start making money from it as quickly as possible?
bduddy@reddit
Who knows, but clearly they continued to release products without addressing any of those issues.
A1oso@reddit
The title implies that Wilhelm Schickard intended to scam us with AI in 1623, by inventing the calculator. Most of your points are valid, but the conclusion is just insane.
jmhuer@reddit
It’s a gross oversimplification It’s like saying roundworms invented thinking because they were first to have neurons
Calculator -> LLMs is not a trivial cross
quetzalcoatl-pl@reddit
this. exactly this.
DerelictMan@reddit
Clickbait title
TheBlueArsedFly@reddit
Yeah and the internet will never take off either
Nervous-Cockroach541@reddit
The thing that scares me, is it's easy to spot programming mistakes. Subtle emission of error handling, logical errors, mistaken use of library functions, version mismatching.
But imagine all the other mistakes in fields not as objective as programming that these things are making that go completely unnoticed.
Actual__Wizard@reddit
Yep. You fell for it. It's the most expensive text generation algo theoretically possible. You were so close to figuring it out.
There was technology back in the 1980s that did the exact same thing that LLMs do, but at a tiny faction of the energy expenditure. Unfortunately, the tech didn't work out, but instead of perusing ultra efficient AI tech, they pursued LLM tech instead.
They keep using ultra inefficient techniques that are not reliable in place of techniques that are ultra efficient and reliable.
joe12321@reddit
A counterpoint here is that indeed if you didn't start using a calculator when everyone else was, you were probably left behind. The fear being created MAY come to be seen as prescient. And even if a tool isn't always perfect, you really can't JUST look at the problems caused (and all new tech causes problems), but the problems vs. the benefits.
But more to the point, there is no con here. Victims of cons don't get an upside (or not certainly). LLMs provice a service (warts and all) plus sales/marketing tactics, and though you can use it unwisely, you can get all the upside out of it you want. Not everything that comes with slimy sales tactics is a con.
qruxxurq@reddit
“Left behind” what, exactly?
What a bizarre-o take.
joe12321@reddit
The article made the point that the culture around LLMs claims that if you don't adopt, you'll be left behind. My point, by extending the comparison to mechanical calculators, is that calculators and adding machines and what not did become necessary and if you for some reason were obstinately against them, you would be left behind in that line of work.
So what the author claims is part of a confidence trick, urging people into adopting LLMs, may just be good advice. And in any case it's perfectly reasonable to believe plenty of people giving that advice are sincere in doing so. And while some of them are just employing sales-tactics, due to all of the above it's just way too far from what happens in a genuine con or scam to equate the two things.
giantrhino@reddit
I always describe them as a magic trick. They’re doing something really cool… in some ways way more impressive than what people think, but because they don’t understand what’s actually happening their brains assume it’s something it’s not.
For magic tricks our brains come to the conclusion it’s magic. For LLMs our brains come to the conclusion it’s intelligence/sentience.
HorstGrill@reddit
It's quite different. Do you know the meme with three guys and the bell curve? Well, when you dont understand how LLMs work at all, they are magic. If you think you know whats happening, it's not magic at all, but just an ultra advanced text completion tool, when you really go into depth about how those networks work, they are, again, magic.
I can wholeheartedly suggest the Youtube channel "Welch Labs" of you want to see some awesome visualizations about some of the few things we actually know about LLMs or NNs in general. The latest 4 Videos are 100% awesome.
giantrhino@reddit
I actually recommend the 3blue1brown series on them.
poladermaster@reddit
Honestly, the confidence trick isn't just from the creators, it's from us. We want to believe, because the alternative – facing complex problems ourselves – is harder. It's like relying on 'jugaad' solutions for everything, sometimes it works, sometimes you end up with a burning scooter. But hey, at least it's something.
LavenderDay3544@reddit
True intelligence requires the ability to add, remove, and rewire neurons, change each neuron's membrane potential in real time, have each dendrite transform its input signal in non-linear ways, have absolutely no backpropagation, allow for cycles in neuron wiring to support working memory, and encode signals not only in the output voltage but in the timing of spikes as well.
The current overhyped so called artifical neural networks are an absolute joke in comparison. Oversimplified would be the understatement of the eon. It's glorified autocorrect in comparison to true intelligence which is the aggregate of a large number of different emergent properties of a very sophisticated analog system.
Traditional digital hardware using the von Neumann architecture is fundamentally the wrong tool to even attempt to explore something in the direction of true AGI no matter what Scam Altman and Jensen Huang try to tell you. These corpirate dorks claim we need to build infinite data centers and assloads of nuclear power plants to power them in order to reach AGI but they're lying and they know it. They just want an excuse to prop up their grift for longer and get more free money in the name of their fake AI.
In reality you would need a neuromorphic chip that is similar to an FPGA but with analog artificial neurons instead of CLBs and with a routing fabric that can allow neurons to rewire themselves on the fly and learn things organically through neurons attached to inputs and respond via neurons attached to outputs.
True AGI isn't a bunch of statistics and linear algebra, it's fundamentally an analog electrical engineering problem. And to demonstrate just how wrong the current corporate grift is, look at how much hardware and power they're wasting on their glorified autocorrect and then compare that to a human brain which is incomparably more powerful but operates on only about 20 Watts. That's the difference between their overhyped statistics and matrix toys and wet, squishy, constantly self modifying analog reality.
qruxxurq@reddit
That was a lot of words for “LLMs don’t actually think, you dumdums.”
LavenderDay3544@reddit
That's a massive oversimplification and what I said doesn't apply to LLMs alone. It applies to all so called artificial neural networks which are about as similar to neural tissue as stick figures are to human anatomy.
What you said is indeed entirely self-evident. A bunch of matrix math is by no means anywhere near even simulating the biological underpinnings of cognition. But the key thing to understand about the bullshit the corporate goons are selling is that without replicating cognition there can be no AGI no matter how much you scale up the existing methodologies. They're fundamentally the wrong approach to reaching that goal despite claims to the contrary.
qruxxurq@reddit
Who are you saying this to? I built NNs from scratch in the 90’s. So, it’s pretty damn clear that however you fuck with the linear algebra, it’s not “thinking”.
I would expect all competent programmers to understand this.
So, either you’re writing to inform the woefully uneducated programmers (had to check which sub we were in), or you’re preaching to the choir of educated ones.
In retrospect, I suppose this post was by one of the more informed, given their seeming confusion over the subject. So, I suppose that’s understandable. But look at it again.
It’s a rambling soliloquy spanning data centers, nuclear reactors, Sam Altman, “neuromorphic chips”, FPGAs, and electrical engineering.
I’m absolutely certain that anyone asking: “Hey, does anyone else see that emperor isn’t wearing clothes?” isn’t going to benefit from a discussion of the best ways to create stealth fabrics.
[Not to mention that there are lots of theories of cognition, and I don’t believe analog electronics has anything to do with it. The problem isn’t quantization or discretization. It’s that current mainstream models are wrong.]
FriendlyKillerCroc@reddit
Has this subreddit just devolved into cope for people hoping that their software engineering skills aren't going to be completely irrelevant in 5 or 10 years? Of course the job will always exist for extremely niche areas but the majority of the industry will vanish.
j00cifer@reddit
Because for one thing it’s an incredibly fast-moving target.
Any negative issue LLM has needs to re-evaluated every 6 months. It’s a mistake to make an assessment as if things are now settled.
Before agent mode was made available in everyone’s IDEs about 8 months ago, things were radically different in the SWE world, and that was just 8 months ago.
j00cifer@reddit
From the linked article:
”…Over and over we are told that unless we ride the wave, we will be crushed by it; unless we learn to use these tools now, we will be rendered obsolete; unless we adapt our workplaces and systems to support the LLM’s foibles, we will be outcompeted.”
My suggestion: just don’t use LLM.
If it’s unnecessary, why not just refuse to use it, or use it in a trivial way just to satisfy management?
That is a real question: why don’t you do that?
I think it has a real answer: because I can’t do without that speed now, it puts me behind to give it up. And Iterating over LLM errors is still 100 times faster than iterating over my own errors.
userimpossible@reddit
If you on your own make as much errors as an LLM, I question the quality of your training.
deja-roo@reddit
Yeah as I was reading your comment I was thinking "well, because if everyone else is using it, I'm practically standing still from a productivity perspective".
yenda1@reddit
before opus 4.5 a couple month ago I would have laughed at anyone telling me they let an AI write more than one line in their codebase. Now I have not written a line of code since it came out.
PotaToss@reddit
How long have you been programming?
Bakoro@reddit
Have you talked to many real life human beings IRL?
Have you ever had the opportunity to pursue other people's chain of thought, and been able to get someone's explanation of why they think things or why the do the things they do?
Have you ever met someone who got a fact wrong, never questioned it, and then lived their entire life with erroneous beliefs built on a misunderstanding?
Humans are more like LLMs than almost anyone is comfortable with.
Humans have additional data processing features than just a token prediction mechanism, but humans have almost identical observable behaviors once you start doing things like the split brain experiment.
It's clear we need something like LeCun's JEPA as a grounding agent and for "world reasoning", but basically all the evidence we have says that humans aren't nearly as objective or reliable as we like to believe.
A great deal of humanity's capacity comes from our ability to externalize our thoughts and externalize data processing.
History, psychology, neurology, and machine learning all build a very compelling narrative that we are generally on the right track.
qruxxurq@reddit
No no no.
Some humans are like shitty LLMs. Many, even. But other humans are completely dissimilar to LLMs.
The 98% or so which are like shitty LLMs are the people who LLMs will utterly replace, yet ironically are not afraid of them. In fact, those people will see LLMs as useful, b/c they are measuring LLMs against their own capabilities, and in that evaluation, LLMs are amazing. They’re certainly more “informed” than most humans.
The 2% which are nothing like LLMs are just sitting here laughing b/c they know they’re not replaceable by GPUs and know how flawed it is to think of a large Language Model as an “intelligence”.
Yet, what’s hilarious isn’t how LLMs hallucinate or make shit up. It’s how pathetic most people are, b/c they’re no better than an LLM. What’s scary is that 1) the stupid people see the equally stupid machine, but think it’s intelligent, assuming that they themselves are intelligent to begin with, and 2) that they think the machines have achieved intelligence instead of realizing that they themselves are stupid, and that the machines are only catching up to their own levels of stupidity, just combined with a very large corpus of facts.
IDK WTF “track” you’re talking about, but if it’s “intelligence”, neither you nor LLMs are on the right one.
Bakoro@reddit
And surely you consider yourself one of these 2% Übermensch.
"Everyone is stupid but me" huh?
I don't even have to say anything else here, your absurdity speaks for itself.
qruxxurq@reddit
Let’s just examine one of the minor points.
2% of 8 billion is a hell of a lot of people.
2% of a high school graduating class of 1,000 kids is 20 kids. Think back to your high school. Now, answer these questions:
You have a grade 3, IDH-wild-type astrocytoma that’s not operable and isn’t responding to chemo. How many in your graduating class would you trust to join their clinical trial of targeted immunotherapy?
You’ve run into the ultraviolet catastrophe. How many in your graduating class will pull a Planck or Einstein and develop Quantum Mechanics?
You need a major choral symphony. But you’re cursed. Anyone you hire will go deaf once they start working on it. It will need to be hailed as a masterpiece 200 years after you’ve written it. How many in your graduating class will manage to produce a 9th Symphony?
I’m none of those people, BTW, though prob closest to the first one.
The real question you need to ask yourself is:
Boysoythesoyboy@reddit
Humans are wrong all the time as well, has relying on other people been a 10,000 yese confidence trick?
qruxxurq@reddit
Yes. LOL
Most people are wrong nearly all the time, and their entire lives are just one long con. See: all of politics.
But not everyone. Every once in a while we get a Beethoven or Michelangelo or Einstein, and slightly more often we get real actual human beings who are thoughtful and honest and ethical and intelligent, instead of almost everyone else who is a mindless automaton.
foodeater184@reddit
I don't see it as a con. It's a new technology that needs time to grow and find its fit. As someone with ADHD it helps me execute much faster by remembering what we were working on, giving ideas I wouldn't have thought of, writing tests, debugging, etc. It works for me.
TeeTimeAllTheTime@reddit
Even if they hallucinate often, which depends on the model and the subject you can still not be a fucking idiot and verify things. Sounds like you just want to shit on AI and make assumptions
pt-guzzardo@reddit
At this point, I'm not convinced SOTA LLMs (thinking Gemini 3 and Claude 4.5, I have less experience with OpenAI offerings) are any less reliable than randos on the internet, which is mostly what you'd get if you Googled a question instead. In either case, it's up to you to do due diligence and verify the answer if you're going to be basing any major decisions on it or using code that LLMs or internet randos produce.
MuonManLaserJab@reddit
I'd read this but I recently learned that humans are pretty unreliable
DavidsWorkAccount@reddit
Because they are good enough. Once you learn how to work with the tooling, it's a net productivity boost.
But there's a lot of learning to be done.
AlSweigart@reddit
Classic article on this: The LLMentalist Effect: how chat-based Large Language Models replicate the mechanisms of a psychic’s con
Baldur Bjarnason included this essay in his book, The Intelligence Illusion, which I recommend.
oscarnyc1@reddit
One thing that stood out to me is that we keep conflating usefulness with intelligence.
LLMs are incredibly good at making hard things easier, like summarizing, drafting, translating and recombining. But that’s different from creating something fundamentally new.
I hope in many more years (400 years?) we’ll have systems that actually reason and discover, but it feels like we’re skipping a lot of steps by talking about today’s models as if they’re already on that path.
Philluminati@reddit
Another one of those posts that says "AI do anything" and yet emphasises the fear.
> Why are we happy living with this cognitive dissonance? How do so many companies plan to rely on a tool that is, by design, not reliable?
> humanity has spent four hundred years reinforcing the message that machine answers are the gold standard of accuracy. If your answer doesn’t match the calculator’s, you need to redo your work.^(1)
^(But they are accurate are they not? I mean the math is the math.. I'm not sure what this point is. If the calculator is wrong the manufacturer will fix it.)
Berkyjay@reddit
This is a comedy post. But I was watching it this morning and surprised to hear how life like and warm they make the chat voices sound. Kind of makes more sense why your average person gets sucked into using them. A majority of the people are not discerning and don't bother to take the time to think about this shit. They just want to know where to find the shit they're looking for.
https://www.instagram.com/p/DTVwuFqATfd/?hl=en
versaceblues@reddit
Because the advancements in the past 3-4 year (including tool use, search, and reasoning) have reduced hallucination to the point where these things are often correct AND find you information on quicker than traditional search.
hibbos@reddit
Humans on the other hand, totally reliable
j00cifer@reddit
From the linked article:
”…Over and over we are told that unless we ride the wave, we will be crushed by it; unless we learn to use these tools now, we will be rendered obsolete; unless we adapt our workplaces and systems to support the LLM’s foibles, we will be outcompeted.”
My suggestion: just don’t use LLM.
If it’s unnecessary, why not just refuse to use it, or use it in a trivial way just to satisfy management?
That is a real question: why don’t you do that?
I think it has a real answer: because I can’t do without that speed now, it puts me behind to give it up. And Iterating over LLM errors is still 100 times faster than iterating over my own errors.
ii-___-ii@reddit
This is only partly true, because AI is also being stuffed into places people didn't ask for. I don't want AI overviews whenever I google search. I didn't ask for AI to show up in my email. It's great when we use it intentionally, but sometimes it's not opt in, and it's there whether you like it or not.
Rajacali@reddit
Because of Peter Thiel the biggest snake oil salesman
rfisher@reddit
First wrap your head around why people are so happy to believe other people without actually checking facts. It is unsurprising that they treat LLMs the same. Don't put up with those people, whether it is LLMs or other people that they're too quick to trust.
lionmeetsviking@reddit
Had to scroll way too far for this comment!
I find that poor LLM is roughly 800% more reliable in terms of factual information, than the current US president as an example.
Adventurous-Pin-8408@reddit
That's just a race to the bottom in terms of trust you can put in anything.
This is enshitification of knowledge. The whataboutism does not in any way increase the validity of ai slop, it just means the ambient information is worse.
lionmeetsviking@reddit
Can’t disagree with you on that.
betabot@reddit
I can’t take anyone seriously that says LLMs aren’t intelligent, particularly for software engineering tasks. Either that person’s definition of intelligence is woefully malformed or they’re in utter denial of what these models are capable of.
Are they perfect? No. Do they make stuff up sometimes? Yes. These are features I would also attribute to humans, though.
azhder@reddit
I can’t take seriously anyone that says LLMs are intelligent. So I will not be taking seriously you not taking others seriously.
Now, let’s have some fun. I will show you mine, you show me yours, we’ll measure up each other’s malformity
Intelligence is the ability to use past knowledge and experience in a new way in order to solve a problem or answer a question.
betabot@reddit
https://deepmind.google/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/
Would a LLM discovering new solutions to frontier math problems meet your definition? Because we hit that benchmark almost a year ago.
Hacnar@reddit
You just proved yourself wrong with that link. It clearly shows your lack of understanding of the topic.
betabot@reddit
Care to elaborate at all, or just going to assert I’m wrong?
Hacnar@reddit
I will thank you for providing a good source of information. I don't have to assert anything. What you write is good enough to let other people make a well informed opinion about your statements.
azhder@reddit
No, it will not.
You can’t print a transistor on a silicon waver by hand and we call you intelligent, no? You can’t spread your wings and fly and we call you intelligent, no?
So it’s not about what it can do or not do. If you had to use that as an example, you didn’t understand my definition.
Worse, you didn’t provide your won. I will not be clicking some link to read how others understand something you don’t.
Now, look into a mirror, there you will see someone not providing own definition of intelligence ready to call other people’s understanding of intelligence as malformed.
No use continuing this thread. Bye.
betabot@reddit
Intelligence is the ability to solve problems. Full stop. I thought that was obvious — people start adding all of these predicates to their definition as you did to try to disqualify LLMs while keeping humans in the set.
WorksForMe@reddit
You're describing functional intelligence or the appearance of intelligence. You can't create a definition and say "full stop" when your definition is only partial.
An LLM is not intelligent in the way a human is. It does not form intentions, possess understanding, or have goals of its own. Its outputs are produced by statistical inference over learned representations, not by grasping meaning or reasoning about the world it inhabits. When it solves a problem, it does so by transforming inputs into outputs according to patterns learned during training, without awareness of what the problem is or why a solution is correct.
Human intelligence is grounded in lived experience, embodiment, agency, and the ability to relate concepts to reality and consequences. An LLM lacks those properties, so any similarity is functional and surface-level rather than substantive.
LLMs are just one slice of the definition of intelligence.
betabot@reddit
LLMs possess most of those things — they are requirements for solving a problem. To solve a problem one must understand the problem, so if a LLM can solve a problem it must by definition possess “understanding” and an “awareness” of the task. To solve a problem one must identify the correct sub goals to arrive at a solution, so LLMs by definition possess “goals” and “intentions”.
Intelligence doesn’t require embodiment or an understanding of reality beyond how reality impacts the solution to a problem.
I think you’re more so interested in the idea of consciousness.
We agree that LLMs aren’t conscious, whatever that word means.
WorksForMe@reddit
For fun, I asked ChatGPT if it was intelligent. It's responce:
No.
Under real-world definitions, intelligence implies an agent that can understand, form intentions, learn autonomously from experience, and apply judgement toward goals. That requires internal states with meaning, not just structure.
An LLM does not meet those criteria.
What an LLM does:
Performs statistical pattern matching over language.
Produces outputs by optimising token probabilities.
Mimics reasoning structures without possessing understanding.
Has no goals, beliefs, intentions, awareness, or self-directed learning.
Why it appears intelligent:
Human language encodes reasoning.
The model reproduces the shape of intelligent behaviour.
Many intelligence tests are linguistic, so fluency masquerades as cognition.
What it is instead:
A sophisticated tool for transforming inputs to outputs using learned correlations.
Comparable to a compiler, calculator, or optimiser, not a thinking entity.
So by everyday, philosophical, cognitive-science, and legal definitions: not intelligent. By marketing and casual usage: sometimes described as such, inaccurately.
azhder@reddit
A hammer solves a problem - not intelligent.
You see bow why I wrote no use continuing? You need time to think things over, figure out what you are missing from my definition. Some other day, maybe.
Bye, for the second time. Will remember to mute notifications this time.
betabot@reddit
Hope you get off that high horse before your existing skillset is totally irrelevant.
WTFwhatthehell@reddit
See. Exactly as I predicted.
They will litterally reject all possible observation amd criteria because they know they're being dishonest
WTFwhatthehell@reddit
Of course
but the "llm's can't do anything" crowd are incapable of learning new things or acting like intelligent entities
alien-reject@reddit
If I don’t know coding but the llm can code a whole website for me, I’d say it’s more intelligent than me
azhder@reddit
You would. Doesn’t mean you’re correct.
It’s just regurgitating the same stuff it was fed from Github which in turn has been produced by thousands of people copying the same cookbook examples in many variations.
But that’s because I don’t know how to build a car and a machine can, so I assume the machine isn’t more intelligent than I am, but then again, I might lack the intelligence…
It’s tricky, isn’t it? People will decide what the answer they like is and then find the metric to support it.
K, you can find more of my takes on this under the above thread, so I will stop here. Bye
falconindy@reddit
You really should read the transformer white paper (or get a chat bot to summarize it for you lol). To dumb it down, LLMs generate one word at a time with the next word being a highly statistically likely word to follow. There's no intelligence. No thought. The"thinking" you refer to is just window dressing on top of your input to give you the impression of intelligence. Tech companies want you to think this is more than just text generation because it's an expensive ass product they're trying to sell you.
betabot@reddit
The human brain is just a bunch of firing neurons, is it not? There’s no intelligence, no thought, just biochemical and electric signals.
There’s brain and LLMs obviously work very differently in practice, but I think calling these models next token predictors is reductive to the point of being misleading. To solve a problem one must predict the correct tokens. To predict those tokens one must be truly intelligent.
falconindy@reddit
If I ask you to finish the phrase "the sky is ...", you would answer blue, because you learned this as a fact. The LLM would probably also answer blue because this has been stated ad nauseum throughout the bot's training data by people who have learned the same fact as you. It's well known that you can poison LLMs (especially those using RAG) into giving wrong answers with surprisingly small sample sizes: https://www.anthropic.com/research/small-samples-poison
seaefjaye@reddit
I think it comes down to expectations. If your expectation is that they're flawless then you're going to have a bad time. These were built by people who have flaws and trained on data created by people who have flaws. Unfortunately for many that isn't the expectation, the expectation is that they can mimic human labour, and with that comes an understanding that human labour can have flaws. What you then need to solve is creating systems to mitigate those flaws, similar to systems we have in place to mitigate human mistakes.
If we're comparing humans to these systems, we only have to go as far as the flat earth society to understand that humans knowledge or intelligence can be easily poisoned.
betabot@reddit
The way these models operate and learn is very different than a human. What does it mean for a human to “know” something, though, other than being exposed to that pattern over and over again? If you sometimes showed a human a purple sky there (as is suggested in your poisoning example), then purple would be a correct answer at least some percentage of the time.
These models don’t need to mimic how the brain works to solve problems, though, they just need to predict the next token with high enough accuracy.
mosaic_hops@reddit
LLMs are regurgitating the OUTPUT of millions of human brains that have reasoned and applied actual intelligence to solve problems. LLMs have absolutely zero ability to do this on their own- they can’t understand, they can’t reason, they can’t problem solve. The output from LLMs approximates intelligence - and looks like intelligence- because it was trained on it, NOT because it IS it.
This fundamental misunderstanding of LLMs is dangerous IMO because so many people don’t realize their limitations.
betabot@reddit
What is the difference between mimicking intelligence and being intelligent? I’m being serious. To correctly mimic intelligence you must also be intelligent.
This mimicking argument isn’t even really true anymore with the RL paradigm, though, because these models can often verify their answers and learn without a human generated data source. Learning physics for example in a simulator.
rfisher@reddit
Indeed, the Planet of the Apes novel was based on the idea that maybe intelligence is just an emergent behavior of imitation.
mosaic_hops@reddit
It’s the difference between playing doctor using Google or being a doctor.
IlliterateJedi@reddit
A lot of us have read it. Seeing what reasoning LLMs are capable of has made me reconsider my definition of intelligence and what it means. Watching an LLM iterate from the wrong answer to the right answer by using tools like Python to reconsider and analyze the query is something that to me is by definition 'intelligent' behavior. The same is true with passing an LLM a complete new repository and watching it reason through how it works. It might be built on a scaffold of text embeddings, but the it's clear from using these tools that they are not really just 'guess the next word engines' anymore.
falconindy@reddit
There is no reasoning. No logic. No intelligence. You're describing the effects of adding agentic and RAG capabilities to an LLM for purpose specific enrichment -- enrichment tailored by humans to supplement generative outputs.
hotcornballer@reddit
And you should read the definition of "Emergent behaviour"
Aggravating_Moment78@reddit
Depends on what you use it for as with anything else. It’s good for some purposes, not so grrat for others…
Lothrazar@reddit
nice clickbait
jameson71@reddit
Because the LLMs are tuned to tell the user what they want to hear.
NotUpdated@reddit
The value for me is that it's software (non deterministic) that can produce classic deterministic software -- thus my program will be correct after I get it correct and every time.
Your point lays well in the bucket of those who are letting it generate things directly for users (emails, business questions, customer service, order taking etc...) lots of those have failed in many ways.
It's better to collect the best business questions 50-200 of them, and programmatically create software that answers though correct every time.
cajmorgans@reddit
In theory every developer also has a probability distribution of "% times being right" when f.e coding. If LLMs can match or surpass the mean probability of "writing the correct code" for a developer, it's essentially a tool that is going to increase productivity by ten folds, and it would be stupid to not use it, because it has one big advantage, as it can write code much much faster than any human possibly can.
sloggo@reddit
I think a big factor you’re not computing there is the time it takes to figure out what is right when you’re wrong. When you’ve worked and built every screw and gear in your machine, you’ll have a much better intuition for why it’s not working correctly when it isn’t. When the generated code makes mistakes, you can try and reprompt, and if that doesn’t work you then have to spend longer than you ordinarily would figuring out what’s wrong.
Given the extra overheads it’s not just about matching and surpassing error rates, it has to very significantly surpass error rates.
In practical terms - in my limited experience - I find myself working incredibly faster (maybe 10-20x) and with less cognitive load for like 90% of the work. But then paying a bit of a price solving and getting an understanding of the trickier bits. And it all averages out that I find I’m getting stuff done maybe twice as fast.
InterestingQuoteBird@reddit
Exactly, it is similar to statistical hypothesis tests. There is a profund difference between understanding something and making a mistake and not understanding something and believing you have a correct implementation. Both result in faulty logic but it is much harder to fix it in the second case.
cajmorgans@reddit
You are absolutely right, and this is the biggest issue with this setup.
efvie@reddit
One of the big problems here is that programmers are terrible at that probability calculation (as most humans are) and LLMs are excellent at making you feel like you're accomplishing something through their mode of interaction even when you're not.
Programmers also love technical problems. My guess is that nearly all the effort that isn't just straight-up garbage production is producing a new ecosystem around these supposedly useful tools instead of anything of actual value just like we've spent billions rewriting shit in TS without really fixing any of the core problems in webapp development, only infinitely worse.
Are you shipping faster?
mosaic_hops@reddit
Maybe but writing code has never been the bottleneck for experienced programmers. That’s the mindless, fast and easy part. A monkey can code.
Getting the architecture right is the hard part, and what LLMs produce is terrible in terms of architecture. Not to mention the code is full of race conditions and deadlocks due to incorrect design, severe bugs, incorrect assumptions, other architectural anti-patterns, or it uses deprecated APIs, mixes multiple approaches to a problem instead of choosing one or the other (by, say, using portions of two different libraries that do the same thing more or leas) or simply doesn’t work at all as described. This all adds significant headwinds that, in our experience, mean AI hasn’t sped us up at all.
It CAN be useful for researching problems but the code LLMs produce - that we’ve seen - doesn’t belong anywhere near production.
I think this is partly due to the nature of the code we write - we’re building new things, not just remixing a bunch of existing things. It takes an understanding and the ability to reason to build new things as there’s no training data to regurgitate from.
cajmorgans@reddit
"Getting the architecture right is the hard part, and what LLMs produce is terrible in terms of architecture". It's actually not terrible, as long as you have some kind of reference and idea of what you want to do.
For instance Claude Code plan mode is far from terrible, and it lets you be part of deciding the architecture, based on the problem you describe. Of course, you need to know what the hell you are doing, but using it as a tool for improving your current idea, or just getting it down on paper, with a feedback loop, is very valuable.
RepresentativeAspect@reddit
You’re asking why we’re happy living with “an incredibly powerful tool” that is not perfect and always right?
LLMs are right more often that I am. They are not helpful and accurate always.
jampauroti@reddit
Just because the calculator got invented, doesn't mean maths becomes obsolete
hotcornballer@reddit
Half the articles on here are AI slop, the rest is AI cope. This is the latter.
FlyingBishop@reddit
It can be both.
peligroso@reddit
Plot twist: OPs post has telltales of Gemini copypasta.
LowB0b@reddit
> LLMs are an incredibly powerful tool, that do amazing things.
You should read the article as well. It isn't inherently calling LLMs "bad", it's calling out the hype and manipulation going on around them.
beatlemaniac007@reddit
Agree with the confidence point. But not sure that automatically means they ought to be rejected. Sounds to me like we need to adjust our expectations (which will likely happen organically) as now it's moved from deterministic to probabilistic stuff. It seems more like a transition phase, which will always come with uncertainty and fear.
In general it seems to be in line with how things progress in this industry. Trading control for leverage. When we got C we gained more leverage but gave up control of specifics of memory registers, etc. When we got Java we gave up control of memory management. SQL allowed us to be declarative and not worry about the "how". AI seems to align with this. The main paradigm shift is the probabilistic approach and I don't know if it will stick, but honestly given how much leverage we're getting out of it might just cause us to accept a lot of slop under the hood.
vansterdam_city@reddit
I swear people who bring up hallucinations have tried this stuff a few times early on and then made up their mind.
If you’ve been continuously using the newest models, they are getting significantly better. I honestly haven’t had a straight up completely fabricated hallucination out of GPT 5.2 and I use it every day.
watchfull@reddit
People don’t understand how they really work. They think it’s next to magic and don’t have the bandwidth/time to grasp the scope of the current models/technology.
Valendr0s@reddit
We've built this cool new product. You give it all the answers - the questions have to be specific, but if you ask a question we've programmed in, you will get the right answer every single time. It's called a 'computer'
<50 years later>
Okay guys. You like the computer so much. We've developed a brand new thing. How about if when you ask a question, the computer responded like a person would, all confident and nice... but a large percentage of the time it's just completely wrong?
dummytroll@reddit
"Large percentage of the time" is highly inaccurate
dummytroll@reddit
I guess because the average human probably hallucinate's more, yet we still read reddit posts
khalitko@reddit
It's a tool. Not all tools are perfect.
Thursty@reddit
oadephon@reddit
This is such copium. LLMs are already pretty good. They can make some pretty complex changes to your codebase with few to no bugs. Where do you think this technology will be in 5 years? It's not going to plateau, it's going to get better by magnitudes in all directions.
We're nearing the end of human wage labor. Focusing on the current flaws with LLMs is just copium to avoid addressing the elephant in the room, which is that AI is finally nearing human levels of intelligence after researchers tried for like 70 years.
thecodingart@reddit
You have to be an idiot to post the words you just did…
oadephon@reddit
Nearly all AI experts think we are fewer than 20 years from Artificial Super Intelligence (ASI). You can put your head in the sand all you want, but these changes are coming quickly.
thecodingart@reddit
Oh honey, I work with said AI experts. You fundamentally don’t understand the technology of this is what you’re saying.
oadephon@reddit
Not an argument, honey
thecodingart@reddit
You sure love spreading AI propaganda for someone who clearly doesn’t know anything about it.
I love the posts where you call it profitable on an infinite timeline 🤣
oadephon@reddit
For a guy who thinks I "don't know anything about it," you sure are lacking arguments... Posting Ed Zitron is not an argument, honey
thecodingart@reddit
There’s plenty of info there, but I’m guessing you have a hard time reading. Have AI summarize it for you 🤣
oadephon@reddit
Unfortunately, you're not Ed Zitron and I don't see much point in arguing with somebody who's not here... But if you have an argument or a point you believe, I'd gladly try and debate it with you.
What's adorable is you refuse to make an argument about how the "world" works, and instead start by insulting people you disagree with, and then post a link as if that link has done your thinking for you. What is your point of view? Why will we not reach ASI in 20 years? If you have beliefs, defend them with reasoning.
Another thing: I didn't say that AI companies are profitable on an infinite timescale. I said that the singular models are profitable, so far. A model will make back the money that went into researching, training, and running it. That's a discrete timescale.
In the same way a company might build a new factory, that single factory may be profitable after 2 years, but if the company is now building 10x the number of factories, the company will still be unprofitable as a whole even if the single factory is now profitable.
thecodingart@reddit
It’s not a disagreement when you’re arguing fundamental facts honey.
ppppppla@reddit
Just one more trillion bro then we will have agi bro then nobody will have to work bro it will change the world bro
oadephon@reddit
Bro the technology that's only 3 years old has problems bro, it's NOT gonna get much better bro don't worry
_darth_plagueis@reddit
If you use llms, you should know they allucinate e check things. They save you a lot o time in certain tasks, so it is worth it on a personal level.
If you think about the amount of resources used to produce and maintain llms, probably they are not worth it. They may become more efficient later, we will see.
dmonkey1001@reddit
Like any tool it's only useful if you know how and when to use it.
AutoPanda1096@reddit
I think people overestimate their capability which leads to others deciding that ai is useless.
Yep, I see mistakes and I also see it helping me in ways that nothing else ever has.
I was asking AI about a business spec earlier and I was fantastic at helping me understand enough to be able to find the relevant regulatory guidance.
Ran it by the business users and programmed it in
I was able to do something in minutes that might have taken hours previously.
And it keeps happening.
The trick is to remember that it's just a tool
Ask the right questions.
"Where do I need to look to find"
"What options should I read up on"
"I've been approaching it this way, can you suggest things I might have missed"
And then you take the answer and apply your own intelligence
LLMs don't exist in a vacuum and I think this is the mistake people who struggle to use them effectively are making.
See it as being like a colleague sitting next to you. I sit next to Steve and sometimes he talks crap and sometimes he points me to the right thing. I never trust Steve explicitly because he's fallible. Like any source tbh.
Ask the right questions.
Apply your own intelligence.
I've been doing this job for 30 years and these tools are a step change in my productivity. Do I see stuff that doesn't add up? Hell yeah! Does that make LLMs "a trick"? Hell no.
I used to think like you but slowly and surely I learned how to use the tool more effectively.
Back in the 2000s I leapt ahead of my peers because I could Google better than them.
The same is happening again.
Some of us will use these tools much better because we get what works and what doesn't.
The irony is that it seems you need a degree or intelligence to get the most out of artificial intelligence.
PublicFurryAccount@reddit
You're absolutely right!
DustinBrett@reddit
Common knowledge is outdated quick when you discuss tech. Things change in months not decades. AI is soon to be Alien Intelligence.
oblong_pickle@reddit
Have you met people? They make mistakes all the time, what's your point?
EntroperZero@reddit
But computers don't, or at least, very, very seldom if a bit is randomly flipped somewhere or an actual hardware bug exists without a known workaround.
Software has bugs, but those can theoretically be discovered and fixed to make the program more correct. The behavior of an LLM doesn't follow this pattern at all, it's just a statistical model that will hallucinate a significant percentage of the time.
None of this is to say LLMs are bad or they can't be useful for anything, but they are a completely different paradigm from what people are used to with computer programs. And no, they're not the first probabilistic computer programs, but they're the first to see such widespread use by people who don't really understand what that means.
syklemil@reddit
People are also generally expected to learn and improve themselves, or else find something else to do.
If a junior produced work at the level of some LLM and never learned (outside some odd growth spurts at rare intervals), that would inform their career options.
Jaggedmallard26@reddit
A threshing machine never learns compared to a farmhand and will snag on the same things every time but its still more efficient than hiring the farmhands. The idea that automation technology needs to be constantly iteratively self improving is a weird one considering the entirety of modern civilisation is built on automation technology that doesn't iteratively self improve.
WaterNerd518@reddit
That’s what they said. The thresher will never move on from being a thresher. They didn’t say something about the inability to learn rendering anyone/ anything useless, just that it’s a fundamental limitation that threshers, like LLMs, are not capable of solving problems, they are only capable of following operator instructions.
syklemil@reddit
Also a fundamental difference between people and tools: People have careers, tools don't. We don't treat limitations in tools and personal limitations the same way.
There's also a huge span in tool usefulness. Comparing LLMs and devs to threshers and farmhands is … likely overselling the LLMs. LLM sellers often come across as about as trustworthy as used car salesmen, or NFT sellers for that matter.
darraghor@reddit
the point is you pay people (developers) for the outcome of no mistakes, in software you collaborate and/or test your work and correct mistakes before shipping.
With AI tools people are often not doing this. They trust the AI tools more than they should. People are used to computers giving deterministic answers. LLMs are different but people haven't adjusted yet.
economic-salami@reddit
AI permanently replaces juniors, who usually do relatively mundane work and are net losses over the short term. Juniors leading juniors get nothing but seniors leading tireless juniors can actually do better.
Fr-Rolfe@reddit
In your haste to blame the tools over the people driving them, you blamed the people driving the tool.
mosaic_hops@reddit
The tools are to blame. If tools make devs 10x more productive (they don’t, but for sake of argument here)- that’s 10x more mistakes (actually more, since the tools have a much higher rate of mistakes). So say 100x more mistakes in the same period of time. This gums up the PR process and it takes time to unpack and validate the messy code LLMs produce.
Fr-Rolfe@reddit
Would you drive a car at ten times the speed of your current car in the same way your present one? Would it be your fault or that car's? A person who loses their leg to a chainsaw when only trained on a carving knife can blame the chainsaw if they want, I suppose.
I don't think AI is the panacea. Hardest part of my job has always been getting people who want me to build something to know what they want built. Describing it to an AI when no fscker knows what they want didn't get better.
If the problem is the tool has produced a behaviour in the user of the tool, that fault lies with the user of the tool.
Sparaucchio@reddit
True
Who does that, i think the industry has stopped shipping bug-free products 20 years ago
baronoffeces@reddit
Replace LLMs with religions in that post
drodo2002@reddit
Well put.. inherent expectations from machine is precision, better than human. However, LLMs are not built for precision.
I had posted on similar lines someone back..
Prediction Pleasure: The Thrill of Being Right
Trying to figure out what has made LLM so attractive and people hyped, way beyond reality. Human curiosity follows a simple cycle: explore, predict, feel suspense, and win a reward. Our brains light up when we guess correctly, especially when the “how” and “why” remain a mystery, making it feel magical and grabbing our full attention. Even when our guess is wrong, it becomes a challenge to get it right next time. But this curiosity can trap us. We’re drawn to predictions from Nostradamus, astrology, and tarot despite their flaws. Even mostly wrong guesses don’t kill our passion. One right prediction feels like a jackpot, perfectly feeding our confirmation bias and keeping us hooked. Now, reconsider what do we love about LLMs!! The fascination lies in the illusion of intelligence, humans project meaning onto fluent text, mistaking statistical tricks for thought. That psychological hook is why people are amazed, hooked, and hyped beyond reason.
bring_back_the_v10s@reddit
But there's a group of people who think otherwise due to the mentioned 400 years of confidence in precise machines.
MrDangoLife@reddit
citation needed
etrnloptimist@reddit
Every tool has their problems. Doesn't mean they aren't useful. I hate when my IDE doesn't jump to definition, but I don't throw the whole thing away.
databacon@reddit
I wish people who don’t like LLMs would just not use them and stop writing articles about it.
scandii@reddit
usually I just click out of every blog post about LLM:s in the first paragraph because they're genuinely a boring read with lukewarm ideas being expressed but this was a pleasant read - kudos!
shokuninstudio@reddit
My position is that I take LLMs for what they are. Just another family software with bugs. That means I don't expect them to be anywhere near the hype levels claimed. Even the best models are often wrong and gaslight you until you give them a reference to prove they are wrong. Then they apologise profusely as if they really feel sorry.
cavedave@reddit
I don't see where the 400 year figure is from. For a 60 year reason the Eliza Effect. Weizenbaum wow the first chestnut and people using it forgot it was a program after a few minutes.
SwoopsFromAbove@reddit (OP)
Mechanical calculators were first invented in 1623. Since then we’ve built a whole society on the idea that computer-generated answers are always accurate and reliable.
LLMs behave fundamentally differently, but their vendors exploit these societal expectations of perfect accuracy, along with other manipulative framings like “friendly” conversation tone and developers’ fear of falling behind, to drive the hype cycle and astronomical investment.
syklemil@reddit
And to an extent that's beyond anything in the field would think, with maybe the ur-example being given by Charles Babbage: