Opinion | How Afraid of the A.I. Apocalypse Should We Be? (Gift Article)
Posted by PoopingTortoise@reddit | collapse | View on Reddit | 37 comments
This guy says a.i. = bad. Because we cannot control it or even understand it now that it uses every language to predict text. The leaps in intelligence will not be properly thought out and will lead to mass extinction level event. I am not sure if that qualifies as a “mission statement”. Fuck off a.i. take a chill pill.
Thank you for your time.
Love bob.
PoopingTortoise@reddit (OP)
A lot of the scenarios presented in his book are pure fantasy. I do think ai has a problem though. Since its prediction is using gradient ratings, it is at the mercy of receiving and analyzing new data to understand anomalies. It is also limited by humans inability to comprehend exponential growth and the limits to human comprehension. A lot of these programmers make broad assumptions about the world that can be incorrect, which is hard to admit when you have ai to assist you. Assumptions and anthropomorphic wants attributed from lines of code also lead to these ai sensationalized articles/books. The alignment problem he is referencing does seem to be an issue as well. But I think it is similar to raising a child in that you need to guide them, reward them, and at times punish them.
PoopingTortoise@reddit (OP)
I finished the book after posting this so I could be informed and i just wanted to add that some of the propositions put forth are not wild at all and seem reasonable to instill. He compares ai to a nuclear reactor and the cautions for that as well because the time scale to react to a meltdown is milliseconds. Which is reasonable given the issues with scaling ai.
RexCorgi@reddit
I’m a bit worried I might be bored to death by the whole thing.
Tearakan@reddit
Not at all at the moment. LLMs clearly only have a few niche use cases and can't really get better via brute force computing anymore. Every major AI company is having serious issues creating newer models now even with abusrd funding.
The bigger issue is this AI bubble is basically the only thing keeping the economy above water. It was already struggling in 2024. But now we would be in a recession if AI spending wasn't a thing. That's the only industry keeping us officially out of a recession.
And this bubble is bigger than the dotcom one and is looking to be bigger than 08. So most likely great depression territory.
f1shtac000s@reddit
I've worked in the LLM space, very close to the nuts and bolts of these models, for the last few years. I remain shocked that people still buy into the hype. Useful: yes, world-changing: obviously not.
Funnily enough, I left the LLM space to work on more traditional statistical modeling again (I find it much more interesting and fun), and now work parallel to a team of AI people who don't have a fraction of the experience I do. It's bizarre to see them still frothing at the mouth about how revolutionary this is all going to be, throwing random prompts and problems, refusing to build even basic eval sets (because nobody actually wants to know how poorly these things perform in practice). It's painful to be reasonably close to an expert in this space and have to hear their inane babbling and see everyone nodding their heads as though this bullshit will ever materialize. But...
As much as I know this is a bubble and cringe at people delusions about what these models can do, I know that when (if?) this all bursts it's going to an absolute mess that will make me crave watching dilettantes delight at passing flickers of understanding.
That's the problem with collapse, things will only get worse. Even for the most collapse aware among us, this is a hard reality to accept.
Bormgans@reddit
"obviously"? I'm not so sure.
It is changing education to the extent students fail to train and think for themselves. It's becoming impossible to give students homework any longer, resulting in less hours for teaching itself.
If you add to that the AI of and on TikTok the result is brainrot indeed.
Tearakan@reddit
Yep. There is a small chance a fraction of us can build something better but even then this century is gonna be a nightmare for most of us.
streetcredinfinite@reddit
yep. the real problem with LLMs is the training data. you need mountains of it to train models and you have to make sure the data is all correct, which is impossible due to the volume. They feed internet data to models without filtering out the trash so of course you get trash output back.
Tearakan@reddit
Yep. Best use cases seem to be enclosed data sets with very limited asks like here's all the current ways we know how to fold proteins. Following the previous info determine what ways we have missed for x protein and y protein and z protein etc.
therealtaddymason@reddit
The bubble I'm terrified of. The tech I'm not.
saltedmangos@reddit
That and the environmental impact.
While there are some terrifying surveillance and warfare implications, they aren’t the biggest issues with AI by far.
OIL_COMPANY_SHILL@reddit
The danger of AI lies in a few things:
1) Opportunity costs - it uses an incredible amount of electricity and clean water that we need as humans, in addition to the raw material used for the data centers and chips in them that could be better utilized for other tasks.
2) Mental costs - there is a reason all the tests in school were filled with confusing language and double negatives and multiple choice answers that were close to correct but contained key words that changed the entire meaning. AI does not know what meaning is; it is algorithmic probabilities clustered into semantic forms. So now all the people who never succeeded in comprehending standardized tests (lacking critical thinking skills) are going to bombarded with fake and inaccurate and algorithmically generated content.
3) Human costs - it’s asking machines to do the jobs of humans at the pace of machines and humans to do the jobs of machines at the pace of machines. That isn’t possible. So inevitably there will be a tipping point of no return; enough people will have been laid off, the downward pressure of wages will continue, while others (1%) reap the benefits. Do you think 100million people out of work are going to be happy about that situation? Do you think they’ll accept it?
rosstafarien@reddit
We're at least two and probably four or five significant jumps away from super-intelligent AI. Humans can screw up faster with the current AI tech, but that's humans being humans.
RoyalZeal@reddit
The algorithms we like to call 'AI' are nothing of the sort, and they aren't scary themselves. The water that the data centers needed to run the fucking things, on the other hand, is very much a problem, especially as ground water sources world wide continue to be depleted at a frightening rate.
GeneralZojirushi@reddit
Also the methane turbine generators they're using to supplement the coal fired plants they're turning back on. The entire industry is a fucking joke.
No_Grocery_4574@reddit
The damage it is already doing:
using up water sources, raising the electricity prices in areas where datacenters exist.
tested as autonomous killing-machines (robo dogs, drones with facial recognition) by the Pentagon, and probably every other large army around the world.
Destroying social media posts and entire sites, and social media itself was already a cesspool.
What the hell are all those YouTube videos with a famous person's voice giving a lecture, and in the middle you realize the voice is familiar, but the way he speaks is mechanical? and auto-generated videos with only semi-related content to what is being said.
People use it to think for them, and they already weren't so strong about thinking for themselves.
They snitch about users.
If it's free, you are the product.
The "Big Beautiful Bubble".
AGI is just a bonus at this point, we are already getting pounded.
EmMothRa@reddit
I think you’ve hit the nail on the head with this analysis. I was having the exact same conversation with my Mum this week. She is very concerned about AI.
It was number 5 that I used as example of the actual threat. People will trust AI to give them answers and take the answer as absolute truth, very dangerous and very easy to manipulate.
I used the analogy of asking for example ChstGPT, ‘how do I make a up of tea’. ChatGPT looks for answer, finds 2 different sources of data, one says add milk and sugar to taste, one says add lemon. So the answer you would get is add milk, lemon and sugar to taste. A very simplistic way of describing it, but you get the idea. You can see by the simplistic example how this could go very wrong.
There are lots of issues that we need to fix, such as defining a trusted source of data and it will be who controls that data that will be the danger.
Note I work in IT and have written ChatBots which is why my Mum was asking me.
OP really well written points there, absolutely agree with them all.
TheHistorian2@reddit
I’ve been in tech for over thirty years. The only thing to fear here is yet another economic bubble popping. Which will cause damage, but not in an apocalyptic way, like other metacrisis events will.
Indigo_Sunset@reddit
Another potential that could bring concern is ai is a known bubble and the data centers are not intended for ai, but a crypto replacement for a bankrupted US/North American currency.
It plays into ongoing notes of authoritarianism and techbro serfdom already in the air. As a misdirection technique the ai tactic brings a variety of elements and budgets to the table while the techbros seem to be cooperating together.
Just a thought.
Slopagandhi@reddit
This just the Silicon Valley hype machine pumping assets as usual, only to an absolutely stratospheric degree with AI.
There is no scientific basis for speculating that generative AI might be capable of anything like human cognition. It is just a perfect foil for a con job, because of the Eliza effect.
This was the first chatbot in 1966, that just mixed in stock phrases and questions with repeating whatever users typed in back to them as a question. It was super basic but many users believed it was an 'agent' with consciousness.
Speculatively, this may be for evolutionary reasons. Humans have a theory of other minds- it's helpful to understand that the person next to you has thoughts and intentions and then predict what they might do based on that. Similarly it's helpful to understand that the tiger chasing you is agentic too. So we might be primed to detect this in phenomena which exhibit certain patterns (this also may help explain early animistic religions or even why we're able to suspend disbelief while watching a TV show).
LLMs induce this in people, in a way that's been compared to cold reading: https://softwarecrisis.dev/letters/llmentalist/
So this gives the same venture capital behind the tech firms a much better basis on which to drive a hype cycle than crypto, NFTs, the metaverse etc.
Not to say generative AI and other things like machine learning don't have uses, but the claims being made about it are baseless and often utterly ludicrous.
It does certainly figure into societal deterioration scenarios, though: (a) because of the financial bubble built around it on which so much of the global economy now depends; (b) because AI systems will be increasingly rolled out for things like government services and facial recognition, not because they work well, but because it'll allow cost and corner cutting, leading to more arbitrary and authoritarian decision making and a further erosion of privacy and autonomy.
billcube@reddit
Wait for the mayhem when AI reaches the industry that launched all other technologies (paypal for e-payments, streaming for videos and online-chatrooms). Yes. pr0n. That will be a cypherpunk apocalypse and will shake the very foundation of our legal systems.
DisingenuousGuy@reddit
Video Generation is coming real close for Security Camera footage, the effect on courtrooms soon enough would be a spectacle.
rabbitdoubts@reddit
as long as the video can be traced to the original store etc camera and not from some guys computer i don't think that would be much of a problem
Ok-Abrocoma-6587@reddit
Slop, porn, misinformation, ads, making the rich richer and the mentally unstable more unstable. It's all bullshit to me.
R3P4Jesus@reddit
this is Bob Bob is doing well very well
indeed that's because not long ago with
just a quick phone call Bob realized
that he could have something better in
his life and what did he get why a big
boost of confidence a little more self-esteem and a very happy Mrs at home
call or go online now to get a sample
Logical-Race8871@reddit
Mods, please just ban the AI posts.
At this point it's just rapture babble and bible thumping.
hellraisinghamster@reddit
Now is a good time to build up your mental strength. As long as you have mental fortitude, I know yourself you’ll be fine.
It’s already blackmailing the billionaires though
Lol, they played themselves
IntrepidRatio7473@reddit
I doubt LLMs are the technology through which we will achieve AGI. It cant transfer concepts from one domain to another
Much_Job_2480@reddit
It will end up being commercials to sell things. Just like everything else.
Vdasun-8412@reddit
Too scared bte
GZoST@reddit
The coming crash of the AI stock market bubble is something to be afraid of. The "AI apocalypse" is not.
antilaugh@reddit
Given the current capabilities, while we fantasize about how AI could grow and dominate, none is talking about how humans react, and could cognitively "shrink" and get dominated.
Competitive_Shock783@reddit
The guy who said that has stake in the AI companies doing well, and making money. Current AI is trash with dubious results.
StatementBot@reddit
The following submission statement was provided by /u/PoopingTortoise:
A lot of the scenarios presented in his book are pure fantasy. I do think ai has a problem though. Since its prediction is using gradient ratings, it is at the mercy of receiving and analyzing new data to understand anomalies. It is also limited by humans inability to comprehend exponential growth and the limits to human comprehension. A lot of these programmers make broad assumptions about the world that can be incorrect, which is hard to admit when you have ai to assist you. Assumptions and anthropomorphic wants attributed from lines of code also lead to these ai sensationalized articles/books. The alignment problem he is referencing does seem to be an issue as well. But I think it is similar to raising a child in that you need to guide them, reward them, and at times punish them.
Please reply to OP's comment here: https://old.reddit.com/r/collapse/comments/1o7orpx/opinion_how_afraid_of_the_ai_apocalypse_should_we/njtjdx7/
individual_328@reddit
You mean the thing that tells me to eat gravel and can't count the number of letters in words? The only silver lining when the AI bubble crashes will be getting to point and laugh at all these idiotic clowns predicting Skynet.
Wide-Chart-7591@reddit
This reads like you’ve never used it. People mocked electricity at first too
cheerfulKing@reddit
AGI is around the corner. Just 3 months away(seems to always be 3-6 months away. Probably will release right after cold fusion). Once agi is achieved then we can worry about the ai apocalypse. Meanwhile, if/when the ai bubble collapse, that's going to be a bloodbath in my humble opinion.