If the bubble bursts, what's gonna happen to all those chips?
Posted by freecodeio@reddit | LocalLLaMA | View on Reddit | 157 comments
Will they become cheap? Here's hoping I can have an H200 in my garage for $1500.
Sicarius_The_First@reddit
It's not a typical bubble, just like the space race was not a typical bubble.
The gov is in on it, and the scale is orders of magnitudes larger than ANY bubble, EVER.
What does this mean? It means that the gov will make use of it. We will not have AGI, because AGI is impossible with transformers, but we will have many different things, and gov will keep propping AI.
bigattichouse@reddit
When the 2001 dotCom bubble burst, what happened to all that internet infrastructure? It stayed put and got used. Sometimes they go dark for a while, but they'll get used.
laurentbourrelly@reddit
I was there in 2001.
It's was the Wild West.
Today is different.
Power players are in place.
It's most definitely an overhyped situation, but Nvidia won't be sweating.
Google is fine. Microsoft, Apple & Co. are still watching.
OpenAI can blow up, Perplexity or Anthropic can bust too. IMO it won't change the big picture like it did back when the dotcom blew up.
SkyFeistyLlama8@reddit
Microsoft and Google will come out of this unscathed because they can tack on LLMs and GenAI to all their existing services. OpenAI and Anthropic definitely could blow up, and then Microsoft and Google could pick up expert staff and equipment on the cheap.
CrazyFaithlessness63@reddit
There was a glut of cheap hardware at the time as companies went bankrupt and assets were sold off.
There wasn't a cloud as such then, many companies built and ran their own data centers rather than lease time from someone else's.
This time around I imagine compute prices will drop if demand crashes which might be a good thing competition wise, the drop in cost would make it cheaper to train models for smaller players.
So you might not be able to buy a GPU cheaper but you should be able to lease time on one for a reduced price.
dragon3301@reddit
Yeah but they don't age out like chips
debauchedsloth@reddit
The fiber did, but a lot was sold for cheap. As for the rest, huge, huge amounts of state of the art hardware was available for super cheap. A lot of it was not economic (either quickly eclipsed by newer stuff or literally too expensive to run compared to newer stuff) but if you needed it and you had the money and ability to use it, it was fantastic. We built a whole startup on the cheap - for those days, anyway. Glorious. We even thought about shifting over to Sparc, because those things were so wildly cheap, but not our wheelhouse.
Also a GREAT time to buy office furniture. If you liked Aeron chairs, you were in hog heaven.
I wonder when the last of the dark fiber got lit? IIRC Google bought a ton - also for cheap - and was just lighting it up as needed for years.
Same thing happened with railroads back in the 1880s. History may not repeat but it rhymes.
I fully expect that to be true shortly, only on a larger scale.
RhubarbSimilar1683@reddit
I did some research on e waste recycling, and modern electronics use so little gold, the copper in the PCBs and the solder is worth 3x to 4x times more
debauchedsloth@reddit
Yeah this was 25 years ago or so. Things have changed for sure.
debauchedsloth@reddit
Nobody is thinking about the overhang, either. Cisco was competing with their old equipment for YEARS, for example.
No_Turn5018@reddit
I don't know if you weren't there or just were very bad at finding deals, but literally almost none of it stayed and almost none of it got used as intended.
bigattichouse@reddit
Fiber optic infrastructure laid dark for a long time, but that stuff is twinkling away now.
No_Turn5018@reddit
Yeah, ALMOST none of it. Everybody on Reddit wants pretend they don't understand how the English language works, for fuck's sake.
Now do you want to talk about servers and parts and the various desktops that the non-it people were using? All the peripherals?Probably 600 other things I'm not thinking of? Or do you want to keep repeating the one exception?
bigattichouse@reddit
I'm specifically talking about infrastructure.
No_Turn5018@reddit
Don't be that guy.
rc_ym@reddit
Bruh. Everyone firesalled (firesold?) everything. So many Sun systems, Cisco switches, Viao's and Thinkpads. The backlog lasted until the mid 00's.
Now it might have been a bit different in the SF Bay Area, but there were so many resellers and cheap equipment.
bigattichouse@reddit
Where I am (Illinois), those firesales led to other businesses buying stuff on the cheap and using it to build the next wave of businesses. It didn't vanish (like I said), it still existed, and it got used to build other stuff.
ASYMT0TIC@reddit
Some of my friends went and bought big network routers on ebay for pennies - level 3 blade-style switches with 100 ports each, fiber modules, etc... they used them for LAN parties.
taylorwilsdon@reddit
Those are always available on eBay for pennies, once they’re out of data center service they’re essentially worthless because they are jet engine loud, consume insane amounts of power and put off tons of heat. You can buy rack mount / blade 40gbps and 100gbps switches for less than $50 right now.
Cergorach@reddit
Especially the old high speed stuff required massive amounts of power = heat. It's one of the reasons why I didn't go 10Gbit Ethernet, the switches at the time were drawing way to much power, as were many of the hardware you would connect to it. These days it's a bit better, but that tends to be old stuff.
In IT our economical depreciation is around 5 years for server hardware and ifra. Sometimes they are technically deprecated way sooner, but that's kinda rare, but I could see it happen in LLM industry...
recitegod@reddit
catalyst 6000 an o0ven
z3roTO60@reddit
Oooh I would love to have one of these haha
bigh-aus@reddit
If you're looking at SFP+ switches, make the jump to SFP28 if you can. Ignore this comment if you're using RJ45.
SexyAlienHotTubWater@reddit
Internet overhead was waaaaaay cheaper than AI overhead, so the cost of keeping it was much, much lower. The margins on websites were huge compared to AI, which is part of what contributed to the bubble. Infrastructure was essentially negligible.
bigattichouse@reddit
Open models are the bits that will be left over.
sarky-litso@reddit
The energy efficiency aspects make this different
burner_sb@reddit
It was a glorious time for buying lightly used top of the line office chairs, as well as great computer equipment. My biggest regret is not buying one of Kosmo's Vespas.
bigattichouse@reddit
Kicking myself, when one of the startups I was at folded, they said we could take whatever we wanted - and it didn't even cross my mind to wheel that Aeron chair out the door with me until the next day.
Intrepid00@reddit
It depends really on the bankruptcy type and who is holding the assets. If it goes to liquidation it gets auctioned off. A lot of people did get some cheap hardware at the bust.
curious-guy-5529@reddit
I don’t think it’s a bubble in that sense. I see it more like the dot com crash. It eventually made it but the burst was just the market correcting the crazy valuations and investments on an at-the-time immature tech. Any AI bubble burst might change the scenery of leading companies but AI in its true meaning will deliver eventually
LowPressureUsername@reddit
You’re not getting an H200 for $1500. There might be literally one chip that gets YOLOed for that low, the lowest they’ll go is probably about 40k
ja_on@reddit
they will probably get surplus'd down the line when newer more energy efficient models come out and you can get them then. idk why there would be any bubble bursting on the card side since people actually buy them for real money.
OldEffective9726@reddit
they turned into the sahara desert that now you see on the surface of Earth.
ptear@reddit
As if I'll ever be able to afford a high end graphics card again.
Cergorach@reddit
If the bubble bursts, chances are that you can't afford any GPU anymore, as that would probably lead to a recession in the US and as we've seen in the previous recession, that can heavily impact the rest of the world.
Also, H200 is a half a million server (8x GPU), I don't see a $60k card suddenly dropping to $1,500 due to the AI bubble bursting. Globally there are still legitimate uses for that hardware that don't aim to earn back trillions, those will probably buy it up at higher prices then you can afford at that time.
Also... Those things need specialized hardware (interface), use a lot of power and generate a lot of noise. Your garage will be an echo chamber and most of the neighborhood will be awake while your local LLM is chugging through the night... ;)
SexyAlienHotTubWater@reddit
If there's a crash in a particular sector (e.g. GPUs), while it will impact the economy and lower your buying power in general, the sector will crash proportionally way more than the wider economy. If you have 50% less disposable income but GPUs are 95% cheaper, you can buy up.
Cergorach@reddit
Did during the dot-com bubble crash the webservers suddenly become 1/20th of their previous value? Did during the US Housing bubble crash the houses suddenly become 1/20th of their previous value? And during all these, did the value of Nvidia GPUs become 1/20th of their previous value? The housing crash devalued the average housing price nationally by 30%. Even during the crypto crash in 2018 NVidia GPUs did flood the market a bit, but certainly not at 1/20th their value, maybe half.
That makes the unavailable $30k PCI-E H200s still $15k... And that's assuming you could even get access to the primary source, when companies go belly up from that format, the folks salvaging value for the claimants generally wants to sell everything asap in huge batches.
SexyAlienHotTubWater@reddit
Come on man you're not engaging with what I said, you're turning this into a dumb internet argument. Let's not do that! My point is obviously not that GPUs will fall by 95%.
But in answer to your question, that is exactly what happened, specialist devices like network switches were being sold for pennies. General purpose hardware was propped up by wider compute demand, but it was still pretty cheap.
Cergorach@reddit
That the AI bubble implodes does not mean that every AI related company will implode, just the big ones and especially the ones that require investment to operate. Companies like Apple, Amazon, MS, Google don't need cash investments, companies like OpenAI, Anthropic, etc. do.
So whether there's a huge influx of H200 GPUs depends on how many have been made, and how many are needed in the western markets. And we also have wildcards like Musk, if he also implodes financially, that's then 100k-200k H200 GPUs. But when they hit the market, it wouldn't surprise me if they wind up in China or Russia via a detour... And other big companies might see it through the AI bubble popping and will use the opportunity to acquire assets or whole companies. While GPU hardware might not go to a penny on the dollar, stocks might...
I wasn't all that impressed by the cheap networking, it was often old, complex and absolutely overkill. While some networking nerds might want a couple of racks of networking equipment, there was virtually no demand for it. With the GPUs there will be. It's just a matter of how much and how much will become available.
At this stage I don't see there being enough supply to make such a huge dent in price. Maybe in a few years, but by then the H200 is no longer the newest model...
SexyAlienHotTubWater@reddit
They won't be cheap because they'll end up in Russia? I feel like you're just making arguments up on the fly to disagree with me.
An AI crash is the same thing as an AI chip crash. A crash means AI assets are worth less money. The chips are an asset. If there's a crash, those assets are definitionally worth less to investors. Assets going down in value only makes sense if people are selling the asset - definitionally, a crash in chip prices means they're available on the market for that price.
There will be some buyers propping up demand in a scenario like that, but we're talking billions of dollars of investment here, Nvidia is 20% of the US stock market or something insane primarily because of AI - companies that want to encode video aren't going to be providing much price support in a scenario like that, there just isn't the volume of demand outside AI.
bigh-aus@reddit
https://www.newegg.com/p/N82E16888892009
Newegg had H200s PCIe for $30k for a long time.
The bigger question is how much of deployed stuff is DGX vs PCIe.
thehpcdude@reddit
I think what you mean is SXM vs PCIe. I only work on the largest clusters and they are all SXM form factor. You won't be able to plug it into your desktop.
They also require far more power than what a single 20A circuit in your home could provide.
bigh-aus@reddit
Yup you're right - my bad.
Cergorach@reddit
Imagine all the datacenters that are already full, being build, etc. Do you think those are filled with PCI-E or with DGX? I'm betting close to 99% is DGX if they could. It could be that during the period of extreme high demand companies had no choice but to buy PCI-E, maybe...
thehpcdude@reddit
s/DGX/SXM ... There are DGX systems and HGX systems, both use the SXM form factor.
You're not gonna find PCIe cards in training clusters as nvlink and other tech, like the associated IB HCA's, require higher bandwidth.
itsmetherealloki@reddit
Yeah would be nice to see something like that for $2-5k. probably not that low post burst bubble but I’m going to day dream about it and none of you can stop me!!
RhubarbSimilar1683@reddit
There are special cooling racks like dell powercool that can lower noise levels
Cergorach@reddit
I know, but those things are only viable at scale, you need space, money, special infra, power, etc.
Spiritons@reddit
They are building a space station to train AI , companies are not like founded 2 -3 years of history, all of them 20+ experienced actors.
THEY ARE BUILDING A SPACE STATION TO TRAIN FUTURE MODELS !!
SkynetsPussy@reddit
MU/TH/ER 600.
Please please please let that be the name.
ElectronicBend6984@reddit
The fact that everyone is hoping for this tells me the “bubble” isn’t going to burst any time soon. Yes players like OpenAI are leveraged on another planet, but without them the compute infrastructure would still be saturated due to open source utility increasing substantially as time progresses. Chip demand is skyrocketing, not the other way around.
freecodeio@reddit (OP)
When there's talk about bubble bursting, it's already bursting.
It's a slowmotion train crash that you are not noticing. Nobody is trying to replace anything with AI anymore because everyone is running into the same conclusions that it's just unpredictable and unreliable, on top of already having to process your data through unstable third parties that apparently rely on investor money to keep the inference prices the way they are.
In other words, moms asking for recipes isn't going to singlehandendly keep trillions of dollars of money flowing.
RuthlessCriticismAll@reddit
It's not called bursting because it happens in slow motion, lmao.
I'm not aware of any bubble in history which had so many people calling it a bubble on the way up. (obviously it is still possible, of course.)
ElectronicBend6984@reddit
I couldn’t disagree more. Spot DDR5 contracts are up anywhere from 250-400% MoM. That is not indicative of a world giving up on AI utility. It’s exactly the opposite.
darth_chewbacca@reddit
Well it's more computation in general rather than AI, but obviously AI is a massive chunk of the computation industry (half? three quarters? IDK, but a very large chunk).
So while AI demand is causing computation hardware price increases, I think there is also an expectation that the US money printer is about to go bonkers sometime between May 2026 and Sept 2026 in an attempt to stimulate investment and hiring in time for the November election. Buying DDR5 at a 300% mark up right now might turn out to be a 15% savings come August, I think some corpos are making that bet.
ElectronicBend6984@reddit
I completely agree with you. I’m a trader for my day job and imo things are lining up for a period of YCC once this admin gets their Fed pick next May, on top of potential QE in the coming months based on o/n liquidity concerns and private credit losses. Not only would that erode the real value of sovereign debt, but it would also dilute ai capex, serving as a pseudo subsidy for the companies funding the ai infrastructure buildout. That’s just my tinfoil hat take, we know the US admin views this buildout as a matter of national security hence the 10% stake in Intel. PSA I’m not saying I agree or disagree with any of this, I’m just stating the direction I believe we are headed.
darth_chewbacca@reddit
Yup, my tinfoil hat too. You and I are of a shared conspiratorial mind.
When my non-tech friends ask me about an 'AI bubble' I use the short hand of "Trump Bubble" to explain to them that I don't think what we are seeing in the stock market or the rise of tech prices is due to any nefarious actions of these tech giants (though nefarious things might be happening, they aren't the risks we should worry about).
Essentially these tech giants are not "over their skis" as people are want to say, they are firmly planted and the companies have moved into the downhill-tuck, expecting a steep-but-smooth slope on the coming hill. The US administration has out and out said "we are currently grooming the upcoming slope, expect nice fresh powder".
The risk is that the hill actually has some moguls on it (aka the money printer doesn't spur investment nor hiring, it just fucks up inflation expectations), and thus I explain it to my friends as a "trump bubble" even if 'bubble' is the wrong term, its a term they understand.
Mochila-Mochila@reddit
Indeed, the buying frenzy seems to remain in full force. But it's still an arms race between overleveraged companies, so at some point the chickens will come home to roost. Harder will be the fall.
Tired__Dev@reddit
The bubble is a financial speculation bubble, not a the tech is going to go away bubble. People will still need GPUs at the same rate, it’s just the price to earnings of the stock will drop.
There’s still demand for AI, it’s just that stock prices will be more inline with revenue generated.
Healthy-Nebula-3603@reddit
Burst ??
Lol
Maybe you say line crypto? Did crypto lost it's value?
Clear_Anything1232@reddit
Knowing these companies, they might landfill them before letting them flood the market
To maintain margins
Fywq@reddit
Nvidia has a huge stake in a lot of these companies. If those chips ends up on the market in some way it's directly bad for their continued business
smuckola@reddit
Why? That statement reminds me how 1990s unix hardware vendors and Cisco always gave a trade in value on their own obsolete stuff.
tat_tvam_asshole@reddit
let's pretend 4090s are still going for MSRP (is true in some places actually), ok, approximately $1500 for 50% compute and 75% vram for 50% cost of a 5090, plus pcie 4 boards are now cheap af. if the market were suddenly flooded by 4090s, I and probably many people would be willing to cluster those, especially with the possible upside of doubling vram. This in turn depresses the values of latest gen offer because GPU compute demand is already saturated. without a scarcity, the time to roi is longer, which delays them in their race to stay ahead of AMD and Huawei, Nvidia's closest competitors.
Fywq@reddit
Because it's better business for them to sell new stuff? Now if Nvidia can take the GPUs compensation for a their share in a company they can sell it again, but why give random auction winners a trade in bonus? They didn't buy the original equipment from Nvidia in the first place. I would imagine Cisco etc. gave trade ins on original purchases - not second hand?
koflerdavid@reddit
Retreating to the gaming market alone won't save Nvidia. Though they might have just enough money on the side to survive the crash.
RhubarbSimilar1683@reddit
Or buy them back for cheap then resell them when demand appears again, now that Moore's law has slowed
johnfkngzoidberg@reddit
Artificial scarcity.
Innomen@reddit
Right? Clue to the reality in that question. The only "bubble" is around AI. AI it gonna be just fine. This isn't like tulips. It only gets complicated when you factor in that people can act on anything they believe, even if it's total nonsense. The "market" has a LOT of that.
Direct_Turn_1484@reddit
I’ll take some.
The_Speaker@reddit
The Bubble is around LLMs, agentic AI, and the ecosystem of AI, cloud, and data center providers, propping it all up.
The fundamental usefulness of AI in medicine, transportation, robotics, warfare, and communication doesn't go away. The demand for those chips will still be there.
This is a stock market bubble, not the housing crisis.
debauchedsloth@reddit
The demand for the chips will be there, but as newer chips come online, older chips will become uneconomic. If it costs you $4/hr to run a GPU and you can only rent it for $.40 since the newer chips are better (and probably a lot more efficient), those GPUs are dead. Not only will you not run them, you'll dump them. I'm sure that's happening now.
Southern-Chain-6485@reddit
The demand is there... for a price much lower than the price needed for AI developers to even break even, so plenty of them will go bankrupt.
itsmetherealloki@reddit
The dot com crash was a stock market crash. I agree on your assessment mostly. This will be more of a large culling of the heard then a crash to absolute zero. Still should be some fun hardware for cheap in the aftermath but most of the good stuff will get absorbed by the bigger fish.
Tyme4Trouble@reddit
Nividia becomes the most valuable software company overnight? Millions of GPUs that need a new purpose.
In all seriousness most GPUs deployed in the DC today aren’t PCIe cards. Even if they were cheap the minimum design config is 8 and 6-14kW (fully system A100-B200).
Tim_Apple_938@reddit
Why would Nvidia become most valuable software company? What?
RhubarbSimilar1683@reddit
Because of CUDA
Tim_Apple_938@reddit
Why would CUDA be valuable if no one’s using GPUs?
koflerdavid@reddit
Technically CUDA is worthless because it's free. But it ties everyone who has written an application with it to Nvidia hardware (we will see whether efforts such as ZLUDA to run them on other hardware will ever take off) That is a lot of stuff outside of AI.
Tim_Apple_938@reddit
Na. Most people write models using high level abstractions such as PyTorch
koflerdavid@reddit
PyTorch relies on well-optimized backends. Programming against PyTorch makes migrating to other platforms easier, but there remains the matter of writing optimized kernels for them. Also, the stability of the other platforms is quite low because AMD and Intel don't put enough resources into fixing their software stacks. OpenCL is pretty much dead; only Vulkan Compute represent a hope of eventually having a reliable, universal compute API.
Tim_Apple_938@reddit
True but nvidia was like a 100B company before crypto and AI. The gaming and science stuff still there but much smaller amt of the pie
T-VIRUS999@reddit
If there's a market for it, an adapter will be made so they can be used in regular computers
thehpcdude@reddit
I design and build supercomputers for a living. It's not gonna burst. Anyone and everyone is scrambling to get anything they can get their hands on, NVIDIA or otherwise. There's unlimited potential for the compute power. Even if the hyperscalers stopped training new state of the art models, there's still a shit ton of compute required for non-LLM uses.
Traditional HPC never went away and is still in extreme demand.
rc_ym@reddit
The question is if the bubble doesn't burst and the cost of labor hits zero, how many chickens (or OF feet pics) will those H200's be? :)
No-Consequence-1779@reddit
What’s your ip address?
makoto_snkw@reddit
IMO, AI won't go away or "died", just like the internet. It will become cheaper, more accessible... And when it's down just like the cloudflare incident a few hours back, everyone will get panicked.
my_byte@reddit
To clarify - "the bubble" refers to stock prices and valuations of companies. Anything infrastructure will keep going and growing. Financial markets and day to day enterprise operations are two different things. Sure - once the valuations of hardly profitable companies collapse, there will be some effects on the economy, but there's more than enough sensible cost saving or value creating use cases to keep the demand for hardware up. Best case we will see the neoclouds and hyperscalers slow down their arms race for the biggest GPU cluster and consumer cards will become more affordable. The pessimist in me is saying China will invade Taiwan next year, leading to a massive shortage of chips.
mycall@reddit
Giving them to US researcher, a wide spectrum like us, would be fabulous (or origin of money = participating countries)
pottitheri@reddit
4-5 year old chips are still extremely relevant. So it will still be market for long. Even if bubble burst demand will be there.
Fheredin@reddit
When the bubble pops, you won't want an H200, anymore. The AI bubble will pop when people start to internalize the hard limits of LLM tech.
Alex_1729@reddit
Bubble bursting doesn't mean AI is going away.
ttkciar@reddit
Literally nobody is saying it would. Previous AI Winters didn't make the technologies of their preceding AI Summers go away, either.
Alex_1729@reddit
It is indicated by the post.
PeachScary413@reddit
Any unused inventory will be destroyed. If they released it on the open market NVDAs profit margins will tank and we can't have that.
Fuck yeah capitalism 👈😎👈
koflerdavid@reddit
There is no open market for data center GPUs outside of enthusiast circles. Even assuming suitable mainboards become available, I can't imagine a lot of home users willing to put a GPU into their living room that consumes more than a kW of power.
PeachScary413@reddit
This is r/LocalLLaMA .. I'm pretty sure all 10 real users in here would like that.
T-VIRUS999@reddit
How do datacenter owners benefit from tanking prices of Nvidia cards (the very cards they will be buying during/after the tanking of said prices)
If anything, flooding the market with cheap compute cards just before a massive upgrade would be very beneficial to datacenter owners
FastDecode1@reddit
That's assuming the market works as intended and there isn't a bunch of corruption and backroom deals going on.
seppe0815@reddit
Hope never die bro
maifee@reddit
"Hope is a good thing, maybe the best of things, and no good thing ever dies"
BasicBelch@reddit
I doubt people going to stop using AI. There will still be inference workloads and there will still be AI datacenters.
New money , new products, R&D, and growth are where the bubble pop will hit hardest.
kevin-she@reddit
I don’t understand, I thought part of the definition of a bubble is that it will burst, so why the if? Please, someone smarter than me, I’m a wrong?
LucidOndine@reddit
It will burst, that is almost a certainty. One cannot continue buying hardware and making data centers without eventually paying for them. Credit swaps between companies based on stock portfolio values and market shares are the fuel in the fire keeping things afloat. WHEN the stock price of companies like Nvidia tank, it will set off a cascade of defaults across all companies that have made their business as part of these meshes.
One could argue that all of these machines that have been used, historically, to build these LLMs could be sold off or repurposed as inference hardware. That is, instead of building new LLM models that represent diminishing returns, if instead of used in training, they could be turned around and sold as a rent based investment. Meaning, the hardware becomes dirt cheap as the market corrects for the actual value of these LLMs.
ttkciar@reddit
You're not wrong. People are just irrationally clinging to the idea that this time will be different, because they hate the idea.
freecodeio@reddit (OP)
because we've reached peak late stage capialism where everyone knows it's a bubble but turns a blind eye for the sake of the economy, in other words, we still have a reality check, it's just the reality changes.
Usual-Orange-4180@reddit
I will buy me a few servers and drown in AI porn… I mean… Research.
bigh-aus@reddit
The bigger question is - are the companies that go under running DGX or PCIe based systems?
I'd love a PCIe H200 card too (assuming it can be power limited to 300W), but I also figure that larger companies are likely to be using DGX based systems. (I get adapters are available, but still). That might have a downwards pressure on the overall datacenter gpu market though, but a lot depends on use case and who's buying.
The thing that makes me sad about a lot of these newer modern systems - they're not suitable for running in homelab environments - due to noise, electricity requirements and cooling. There's plenty of good powerful servers out there that are really home datacenter / colo only.
Heck the new Dell R7715s can have dual 3200W psus, that exceeds the capability of standard outlets in the USA.
ttkciar@reddit
Yep, this. The last AMD Instinct with a PCIe interface was the MI210, which are currently going for $4500. I'm hoping that will come down a lot post-bubble.
There are also MI300A systems out there which are CPU/GPU hybrids in SH5 socket systems, which only pack four processors per system. Those would have to be acquired on a whole-system (or at least motherboard+processor) basis, but I don't think many (or any) would materialize on eBay post-bubble because they're mostly used for HPC, not LLM inference. I don't think the HPC industry is going to be impacted much, if at all.
Then there's the OAM systems, for hosting MI300X, MI450X, etc, which only come eight GPUs to a system, and would also have to be purchased on a whole-system basis. These make up most of the AMD-based LLM infra, and I've been trying to think about how to shoehorn even one into my homelab. Eight MI300X with a peak draw of 750W each comes to 6KW, or about the power of a large lawnmower at max throttle.
Fortunately I expect to have at least until 2027 (maybe even 2029) to figure it out, and probably later. In the meantime I'm mostly just watching MI210 prices.
RhubarbSimilar1683@reddit
No. They are AI wrappers for the most part
Illustrious-Dot-6888@reddit
Lays is going to buy them all at a bargain price and then sell them with a barbecue flavor.
HecateRaven@reddit
🤣
xJbztqp@reddit
Heat our offices and houses
OcelotMadness@reddit
Im coping and hoping server hardware becomes very cheap and I can snag some stuff before the recession. I would very much like a rack and an Epyc server mostly for learning and non-LLM purposes, but also to host an instance of GLM 4.6 for playing AI Dungeon, lol.
Signal_Ad657@reddit
They’ll still run. Beyond that?
🤷♂️
teachersecret@reddit
The old p40 cards and the like ran in those datacenters for a long time before they got dumped for peanuts on the global market as they got swapped out for newer gear. Plenty of those things still running in LLM rigs :).
I imagine the H100 will be similar, some day... but that day might be a bit... further away...
SlowFail2433@reddit
Not necessarily far looking at used a100 prices
teachersecret@reddit
Yeah, I've been eyeballing those. Gonna be interesting as a flood of A100 40 and 80gb variants hit ebay. At the right price, they're an interesting prospect (not yet imho, but soon).
Ok-Internal9317@reddit
the v100s are next, \\then A series and then H series, just wait for another 10 years maybe I'm hopeful
MaCl0wSt@reddit
they will be given to me, and I will build a pillow fort with them.
SamSLS@reddit
I the early 2000s I built my entire server room with pre-owned Dell servers for Pennie’s in the dollar. Yes capital will get redeployed. And that creates opportunity.
slashtom@reddit
There was a good youtube video recently that talked about why AI isn't going to 'bust' but basically it's being propped up by all the major nasdaq companies. OpenAI is investing in AI companies, Microsoft is investing in OpenAI, Nvidia is circle jerking with other AI companies, it's definitely here to stay. Everyone will keep the money flowing to keep the demand flowing.
Let's just say I wouldn't be short on AI
specialsymbol@reddit
VR is finally going to take off
Ok-Internal9317@reddit
[skull emoji]
kidflashonnikes@reddit
Many people are very falsely accusing Nvidia of them going to destroy their cards in the future in the event of a recession. That costs money to do. When the bubble bursts (spoiler - it already has you just haven’t seen the worst of it) chips will be cheaper but not ALL chips. Consumer chips will plummet - while professional cards will not (data centers think h100s etc). I work at one of the largest companies in the world as an AI researcher - we’ve already talked to many GPU providers and have already been given the heads up on the incoming economy changes aka recession. Due to NDAs - all I can say is that cards will not be destroyed and they big boys are 50 steps ahead of you all. You all should be focusing on squeezing as much GPUs into your setups and focusing on generating revenue or maximizing your hobbies while you still have 6-12 months left before the bubble really deflates. I love reading all your posts btw - all of (ai researchers and engineers and some CEOs of labs) all read this sub Reddit
bigh-aus@reddit
I would also assume that the companies who bought GPUs won't let nvidia destroy them vs buy them back / sell them on the open market.
Novel-Mechanic3448@reddit
As someone who actually works at a hyperscaler, if you're an AI researcher you have almost certainly never even seen the inside of a DC. Completely different job. There is a 0% chance you would be involved in any hardware related conversations, especially logistical ones that are confidential.
You thinking the bubble has burst is hilarious to the point of it making me think you are LARPING, or baiting someone like me to come in and correct you. The demand is unfathomable.
kidflashonnikes@reddit
If you actually worked in the space, then you would know that infra and hardware work closely together, and at my level and clearance we aren’t just doing inference and hardware acquisitions - we are also doing real research that requires all GPUs, including CPUs. I don’t need to prove anything. I’m just here to give everyone a warning. Due to NDAs I’m not allowed to say anything else but I can assure you - it’s very much over already. Lmao “hyperscaler”
thehighnotes@reddit
I will sacrifice myself and will accept such chips surplus
cddelgado@reddit
Unpopular take: we aren't in a bubble. We are in time dilation.
Some things AI promises are already happening and are already being applied as promised. The things business owners want are not there yet because their eyes are bigger than the labor forces ability to deliver--can't automate your entire business if you aren't working with people who know how to automate it. That will eventually hammer itself out.
The other side of this is that the very desire for AI itself--to make it fulfill more of those promises--requires innovation. The tech industry overall stagnated a bit because we hit walls with traditional manufacturing and scaling, and didn't have the drive to push past it. Businesses could get away with re-packaging old stuff in new ways.
That isn't the case anymore. Now we have a genuine need to push past where we are.
All of that takes time. And it takes longer to percolate than the people at the helm of the S&P 500 and the NASDAQ want it to. We are thus in a state of waiting for it to end--Tech Sector Edging, if you will.
Decent-Vermicelli232@reddit
Cheap DDR5 PLEASE!!!!!!!!!!!!!!!!
pmttyji@reddit
I'm not that greedy, but I would like to get one or two 96GB GPU(RTX Pro 6000 maybe) for $1500 each. Any possibility folks?
StupidityCanFly@reddit
We’ll buy some fish and have a feast.
thetwopaths@reddit
Mine mine mine! All mine!
Neomadra2@reddit
That you hope for a crash like a vulture maybe is a sign that there's no bubble? Demand is obviously still going strong
jeffwadsworth@reddit
Good luck. The power required to run it alone coupled with the fact that you might see it in a decade will be akin to using a smartwatch from the 80s.
Johnwascn@reddit
When that day comes, I'll buy four H200 cards to raise my computer desk. Mainly, I feel this desk is a bit low, but I don't want to replace it.
Minute_Attempt3063@reddit
Nothing, companies will have use for them
Novel-Mechanic3448@reddit
There is no such thing as idle compute
DustinKli@reddit
The chips will still be used because there's a demand.
The AI bubble bursting has nothing to do with AI demand and AI usage.
It has to do with over-investment of hundreds of billions of dollars in unprofitable companies and companies that don't even have a product yet.
The internet bubble bursting didn't mean people used the internet less. It just meat all the investments into nonsense went belly up.
a_beautiful_rhind@reddit
Buybacks so probably not. Maybe A100s finally drop in price though. Companies that aren't bubbled may upgrade.
Downside is you get no new models.
Irisi11111@reddit
You shouldn’t stress too much about the infrastructure, even if the capital bubble bursts. LLMs are still incredibly useful in our daily work and life. Resources will simply shift to more efficient areas, allowing capable people to generate more value. I’m open to investing in used GPUs to help create that value.
ak_sys@reddit
What's going to happen to all those chips?
The same exact thing that happened to them when the crypto bubble burst. Nothing.
For the last decade+ compute has been the most valuable resource you can produce. AI hype may be a bubble, but the investment into GPUs is not. If AI wasn't on the table right now, there would be half dozen viable, monetizable uses that those chips could transition to RIGHT NOW.
When/if AI hype dies down, NVidia may hurt for a little bit, but the chip market themselves won't change much in my opinion. The next thing will make AI hyperscaling look small in comparison.
Jester_Hopper_pot@reddit
It depends but the best case is that they are sold for near loss, but if it craters retail it could be worrying in general since no one can keep their doors open while trying to clear inventory
Long_comment_san@reddit
Nothing will burst because datacenters will just grind a different type of data. For example, shifting from LLM chatbots to scientific calculations or game engine physics development. It is hardware first, you can always repurpose the software
butihardlyknowher@reddit
or recommendation algos.
Django_McFly@reddit
i think the bubble bursting is that overvalued companies lose a lot of value and potentially never see their stock return to those levels. how the internet bubble was.
i don't think the bubble bursting is and now nobody will use AI. it's going to go the way of the hoola hoop. people didn't stop using the internet. has internet usage ever had a down year?
usernameplshere@reddit
I mean, these are still businesses run by greedy business administrators, I doubt it will ever get that cheap. They would rather let them rot somewhere than throw them on the market for adequate prices.
Kooky-Somewhere-2883@reddit
it will not, humanity will live in enternal happiness
whatever462672@reddit
Microsoft might dump all the cards they have been storing due to lack of electricity. 🫣
SpaceToaster@reddit
They are orders. We are unlikely to have a glut of chips available for a while, but the orders are going to be canceled in droves.
BumblebeeParty6389@reddit
Whether "bubble" bursts or not, nothing will be cheap for a long time. I can tell you that much
mr_house7@reddit
Sales everywhere
ImaginaryRea1ity@reddit
Same.
grufftech@reddit
the bubble is the stock price, not the physical hardware.
forthepeople2028@reddit
The data centers become inexpensive, but not for individuals. Think of when telecom infrastructure grew at a pace way faster than demand. Idk if you’ll be able to get an H100 for $1500 but you may be able to get an entire data center half off what it cost to build today
MitsotakiShogun@reddit
Cheap? They won't even be able to give H100s for free.