After a lot of deliberation I bought an M4 Max MacBook Pro 128GB RAM / 2TB SSD about 2 month ago, mainly because I was worried it won't be available later. 256GB M3 Ultra Mac Studio delivery times were 4-5 months and the 512GB model was already removed from Apple's website.
I imagined a future, where Anthropic servers are so unreliable, they have to jack up their prices so much, that many companies won't be willing to pay for it.
If I'm already hooked on having such intelligence at hand, it would completely kneecap my ability to program anymore without it.
Qwen3.6 27B FP16 runs at <10 token per sec.
It's painful slow, BUT still faster than me and more convenient than doing things the traditional way and it has intelligence comparable to Opus 4.5 / Sonnet 4.6 low effort.
It opens up the possibility for me to work for companies, who can't afford to provide me good work computers and expensive AI subscriptions, but provide other benefits instead, e.g. flexible working hours, remote work, ability to program in Clojure, not against test-driven development and pair-programming, etc
I just want to point out that some of the hardware we have at home right now is legitimately stronger than building sized supercomputers.
20 years ago, the world strongest super computer was IBM Bluegene/L which reached a peak performance of 480 TFLOPS
today, a single GeForce RTX 5090 can get 104 TFLOPS on FP32, and a whopping 800 TFLOPS on tensor operations. a system with 4 GPUS can reach a similar performance to that.
Ofcourse this is a bit of Apples to oranges comparison, especially that both of these ran different software and BlueGene had a 32TB or ram. but still, What we have now is definitely super computer class machines by 2006 definition.
and blugene was hard as hell to program. basically, code was written in fortran and you had to identify your most intensive kernels/functions via profiling and then optimize it for the double FPU , manually. like almost assembly looking code. weeks/months of tuning. each compute node also only had access to 1Gb/s , share with 32 others, so data management was important.
I hope we get another year or two before it happens. Partly because we might get some better open models out of it, mainly because my savings need to grow a bit first lol
Yeah it's fun to dream and all but these server-grade GPUs for cheap on ebay is never gonna happen. When the bubble pops big companies will scoop up the failing smaller companies and the big companies will literally rather burn the GPUs in a big bonfire than handing them out for cheap and risk any future competition or subscriber churn.
The "cloud" playbook is about long-term global infrastructure monopoly and forcing everybody else to rent everything and nothing else... Selling expensive things for cheap to John Doe to recoup a few thousands makes no sense through that lens
Although the last gen of cloud GPUs (p40s, v100s, things like that) eventually ended up dumped on the open market at scrap values. Things with value tend to end up on the secondary market eventually. A100 are already all over ebay (albeit at high prices). Give it a few years and I bet A100/H100 are available at scale and relatively inexpensively.
To stay with the "PC" only (no notebooks) I have 4x Dell Optiplex 7060 Micro. The configs are i5-8500T (6c/6t), 32GB DDR4-2666 and 256GB SSD. This would have been datacenter rack level hardware 20-25 years ago.
Fun. Was legacy at the time, but I got to briefly work on a 5 mb hard drive with open air platters that were actually platter sized (as in you could have actually serve a turkey on one of them). Buddy kept it in his house. He worked on cat scanners and it came off of one that we retired.
The sad reality is that those that do not understand tech will begin targeting people like us that have home labs. I already had it happen to me, twice.
Oh, I couldn't do that. I let them believe whatever they want, doesn't affect me, until they start sharing to my closest family. I interject, debunk, and that solves it. I just engaged once or twice, didn't have to any more. They have better things to do that talk about that shit, but I guess there are all kinds of people with nothing better to do.
Huh now that you mention it a friend of mine literally did build a server with a huge overkill server rack, UPS, and so on for datahoarding like 8 years ago... not in the same ballpark since he only spend like $4k on it all I think but still.
I built a Self‑healing, self‑learning, sovereign AI development engine
Runs offline on a laptop or tablet. No cloud, no subscription, forever.
Need collaborators to stress test & offer feedback
People have been running “data centers” in their homes since the Opteron existed. Now there’s a forum for people to share their experiences which highlights this and the guy thinks its not normal to him? This happens to people who never stepped out of circle in life.
well if they knew how big some models were then they would realize that isn't an insane setup at all. or that any server mobo having 256-512gb of ram is pretty typical.
It's nothing new. People used to build mining farms much bigger than this.
Heck, 128Gb RAM doesn't sound outlandish at all these days, it's mere 4 times the minimal comfortable number these days, and 64Gb doesn't seem overkill these days. The only reason people started paying attention is the price spike. RAM used to be dirt cheap before MoE models going widespread.
But yeh, I was originally like, I will spend $500 on this hobby. It aligns with my work, and my PhD studies. Plus, cool.
But you are literally making things that think, that you can talk to, that can do productive things for you. People spend $5k on a TV or a sound system. Or $50k on a car. But this, this emulates actual thinking.
$500 doesn't cover that amount. 4 GPUs with that size of money is crazy expensive by average people standards. I can't even justify a single 5090 with todays pricing.
Some people here are crazy rich, "I have a 2x H100 in a theeadripper, should I up it with another two?" Or similar.
Yeah and it's so depressing seeing someone with that power setup go "So what do I do now guys?? Do I install Ollama or something lol I have no idea"... 💀
$500 didn't cover that amount. That is how much I wanted to originally spend. Setup an old machine with a cheap Mi50..
Then I bought multiple Dual socket xeons, Tb of ram and a palette of Mi50s.. Now I am contemplating re-wiring the house. But everything I bought is now basically worth double what I paid for it. And arguably, it isn't going obsolete as quick as old stuff used to.
My idea or dream is that I can automate much of my work, locally using my own systems. And that I learn about AI in the process.
How much do people spend on getting a degree or for a business.
it is funny how people think about these things. if you buy a $50 K boat, that you use a few weekends a summer people would consider this reasonable, but get a $10K computer that you use several hours daily and that feels crazy.
I am even guilty of that, I really struggled to justify to myself buying an $3500 MacBook that I literally use for everything for 3 years nona will continue to use for probably another 2, but it took me a few seconds to justify buying $6000 worth of couches for a living room we rarely sit in.
I just blew a car's worth of money on AI and I have to keep explaining to myself that this is infinitely more important than a car to me. Weird I have to tell this even to myself. We get stuck in outdated value paradigms even after circumstances change.
A computer is a car of the mind, and doesn't nearly as much fuel for the freedom you get. If you use a PC nearly all day, it is probably one of the cheapest investments you can make.
Once AI is good enough to work as as effective agents for any digital task, it would be like having a personal lawyer, accountant, entertainer, news host, and educator for $10 bucks a day. A personal AI computer will be just as fundamental as owning a house.
I had to explain to same to others when they ask me about pcs and MacBooks and then talk about cost. I quickly go to examples like yours and then tell them to stop cheaping out or do, you will use it every day and deal with it anyways.
If he was talking about people with literal datacenter stuff in house, no, it would even be a good wakeup call for some.
But the example is hilarious... Like multi GPU or 128gb of ram was a lot... It more feels like a laptop folk or an envious one that can buy the stuff we want tbh.
Bro.. you stick with your GF's, fake tan and and peptide injections... and I'll stick with my 512GB RAM and way more cards.... 128GB? You must be talking about my laptop and client PC's?
I hope we are building personally owned data centers. IMO, corporations shouldn't own compute nor robots - they should be renting those things from individual households. This would make it much harder for corporations to do evil and stupid things, if they have to answer to actual people.
Democracy, requires many individuals to have a slice of power and no one controlling a large part of the pie.
LLMs perform valuable work, and by a projection there is a payback period for an investment in LLM. For Western frontier model, I would probably use much more than $10 per day, and for a Chinese model, around $5 per day. Depending on the prices and quantity of mechanical intellectual labor you use, you'll probably discover that you're doing more than $1000 worth of AI compute each year. To me, it's pretty much a nobrainer that you'd put in little money to get lots of value back.
Back in the day when you could get the ASUS GB10 for like $2000-3000 price, it was relatively cheap considering the value. Hell, I should have bought two, probably. I'd be running 256 GB range models with decent enough compute and bandwidth today. I regret not doing it now, but I had just bought Strix Halo box earlier that year with 128 GB and I felt weird about buying yet another 128 GB box for more AI stuff.
The real stuff for multiuser remains very costly -- $100k and up is in the ballpark for serving an organization, though with dozens to few hundred users, the math fundamentally doesn't change very much. We can probably bring the insane compute requirement in prefill down in future models, and turn the sequential inference into more like block inference with approaches such as dflash. This way, the models better match the limitations of common hardware, and these consumer boxes many of us have today become more useful.
That's true, even if we don't have powerful rigs. The compute and inference happen always somewhere. It can be your phone, your laptop, or a 4x RTX rig.
It's just that datacenters are far more energy-efficient than any home or office setup.
Do people not realize the issue with "datacenters" isn't that its a datacenter existing, its that its loud and over-uses local and rare global resources and provides virtually no ongoing jobs. There'd be no issue if they didn't effect others, the point is it does and they dont care because it makes them money at our expense. Having your own little home datacenter paying your own electric bill and having to hear fans in your own house that doesn't effect anyone else is a totally different thing.
Idk, when you're the average person looking to get into LLM, DetectiveNarrow's post sums up exactly how it feels trying to get into it lol
It feels hopelessly lost and the people telling you how wonderful local is are also the ones with the hardware to actually be able to enjoy it. Meanwhile, you'll realistically end up trying to run a much smaller model and if you've ever tried LLMs before it won't be as enjoyable as the API models.
Of course, usebase will have this opinion vary greatly. Point still stands.
Truly, I treated myself to a 5090 PC and can run what I want to run but even then some of the coin people are dropping for this stuff is wild. I gets a hobby but damn
I mean It Is true but at the same time people are forgetting the difference of scale. Sure If I have 3 5090's I'm going to be using a lot of power but It's going to be basically nothing when compared to even one of the racks In a data centre
A datacenter in my home was my intention all along. Couldn't be happier. You can't believe how much you can get done with a small army of qwen 35b agents.
2x5090, 1x3090, 2 24 core pc with total 352 gb ram ddr5, 250tb total san space 10gbit network... Will purchase rtx6000 pro end of summer. Just a hobby, but I'm a programmer (still) so I use it for multiple purpose including not paying for a claude with qwen3.6 27b Q8
They look at me weird when I explain my 'data center' uses, on average, less than the power of a 60W light bulb (M3 Max with 128GB) each day. I am currently running DS4 Flash Q2 with 768K of context on it. Not fast, but much faster than my brain for what I use it for (coding assistant, security reviews, etc).
I kill firstborns while their mamas watch. I turn cities into salt. I even, when I feel like it, ^(run 4x 48gb modded 4090s), and from now till kingdom come, the only thing you can count on in your existence is never understanding why.
Outta here with that. Community discussion supporting the community. I get annoyed with people that think they're the neighborhood watch.
99% of people are running something so low power it wouldn't register on the grid. Everything we're doing combined pales in comparison to 1RU in an a AI datacenter
man people been doing this for decades, its only now its something universally interesting unlike overclocking or mining or even gaming which most people grow out off.
spaceduck107@reddit
Damn, I wish I had a datacenter in my home lol. A 5,000 sq ft basement with full racks of GPUs and dedicated HVAC would be sweet.
CreamPitiful4295@reddit
Would you settle for a couple raspberry pi and a box fan?
alphapussycat@reddit
It's more like workstations. If you got 1tb of vram, then sure, I'll yield and call it an at-home-data center.
CreamPitiful4295@reddit
lol! :) “yield”
valuat@reddit
They are all asking the wrong questions. But will it run Crysis?
vagmi@reddit
localhost is just sovereign AI.
onetom@reddit
After a lot of deliberation I bought an M4 Max MacBook Pro 128GB RAM / 2TB SSD about 2 month ago, mainly because I was worried it won't be available later. 256GB M3 Ultra Mac Studio delivery times were 4-5 months and the 512GB model was already removed from Apple's website.
I imagined a future, where Anthropic servers are so unreliable, they have to jack up their prices so much, that many companies won't be willing to pay for it.
If I'm already hooked on having such intelligence at hand, it would completely kneecap my ability to program anymore without it.
Qwen3.6 27B FP16 runs at <10 token per sec. It's painful slow, BUT still faster than me and more convenient than doing things the traditional way and it has intelligence comparable to Opus 4.5 / Sonnet 4.6 low effort.
It opens up the possibility for me to work for companies, who can't afford to provide me good work computers and expensive AI subscriptions, but provide other benefits instead, e.g. flexible working hours, remote work, ability to program in Clojure, not against test-driven development and pair-programming, etc
mohelgamal@reddit
I just want to point out that some of the hardware we have at home right now is legitimately stronger than building sized supercomputers.
20 years ago, the world strongest super computer was IBM Bluegene/L which reached a peak performance of 480 TFLOPS
today, a single GeForce RTX 5090 can get 104 TFLOPS on FP32, and a whopping 800 TFLOPS on tensor operations. a system with 4 GPUS can reach a similar performance to that.
Ofcourse this is a bit of Apples to oranges comparison, especially that both of these ran different software and BlueGene had a 32TB or ram. but still, What we have now is definitely super computer class machines by 2006 definition.
ifdisdendat@reddit
and blugene was hard as hell to program. basically, code was written in fortran and you had to identify your most intensive kernels/functions via profiling and then optimize it for the double FPU , manually. like almost assembly looking code. weeks/months of tuning. each compute node also only had access to 1Gb/s , share with 32 others, so data management was important.
lordlestar@reddit
people where building supercomputers at home using ps3s back then
kmouratidis@reddit
My smartphone has ~250x the GFLOPs of IBM's Deep Blue, and browser/JavaScript stockfish would likely beat it 100-0.
llama-impersonator@reddit
i wish, datacenters have racks of H100/H200/B200
05032-MendicantBias@reddit
I really hope the bubble pops soon, and there will be H100 racks on sale for scrap metal 😃
tim_dude@reddit
After the bubble pops, there will be a lot of foam left
Squidgical@reddit
I hope we get another year or two before it happens. Partly because we might get some better open models out of it, mainly because my savings need to grow a bit first lol
Green-Rule-1292@reddit
Yeah it's fun to dream and all but these server-grade GPUs for cheap on ebay is never gonna happen. When the bubble pops big companies will scoop up the failing smaller companies and the big companies will literally rather burn the GPUs in a big bonfire than handing them out for cheap and risk any future competition or subscriber churn.
The "cloud" playbook is about long-term global infrastructure monopoly and forcing everybody else to rent everything and nothing else... Selling expensive things for cheap to John Doe to recoup a few thousands makes no sense through that lens
teachersecret@reddit
Maybe...
Although the last gen of cloud GPUs (p40s, v100s, things like that) eventually ended up dumped on the open market at scrap values. Things with value tend to end up on the secondary market eventually. A100 are already all over ebay (albeit at high prices). Give it a few years and I bet A100/H100 are available at scale and relatively inexpensively.
In_der_Tat@reddit
What's your RoI and in which productive activities do you employ your metal?
zuraken@reddit
Just like in 2000s scrapped for gold and copper
mc_nu1ll@reddit
you might need a good 15 kilowatts to run a dgx b200 tho iirc, because the corpos have their own power lines and everything
Responsible_Buy_7999@reddit
Why are we screenshotting another post with a drive word question/insult instead of commenting on the post.
Kal-LZ@reddit
He discover r/homelab
k1rika@reddit
Now let him see r/HomeDataCenter/ :D
smallDeltaBigEffect@reddit
hahaha what the fuck
spaceduck107@reddit
:o
Mother_Desk6385@reddit
Now see lowendtalk
brickout@reddit
Um, with respect, you have no inkling of the monstrosities I have built.
Drevicar@reddit
Don’t let someone else tell you how to have fun.
anomaly256@reddit
Bruh... I had a datacentre at home before open weight LLMs even existed
Alex_1729@reddit
Some of us were crazy before it was cool
BidonPomoev@reddit
I did this before it was cool and will do this after it will stop to be cool, will gladly buy used hw from these pretend-to-be-geeks : -)
anomaly256@reddit
I had an 8 node beowulf cluster of 486 machines. (No really, not just saying it for the meme.) Did absolutely nothing with it too.
arbv@reddit
That is the itch I have been having for while - a beowulf cluster from SBCs with absolutely no purpose. Just for the sake of it.
boutell@reddit
Raspberry pi beowulf clusters are a thing https://www-users.york.ac.uk/~mjf5/pi_cluster/Docs/Creating.a.Raspberry.Pi-Based.Beowulf.Cluster_v2.pdf
tmvr@reddit
To stay with the "PC" only (no notebooks) I have 4x Dell Optiplex 7060 Micro. The configs are i5-8500T (6c/6t), 32GB DDR4-2666 and 256GB SSD. This would have been datacenter rack level hardware 20-25 years ago.
letmeinfornow@reddit
I did in the late 90s. Back into it all again. Geek at heart I suppose.
sernamenotdefined@reddit
My father was a network administrator in the 80s. We had IBM PC's and a Netware server at home in the late 80s..
I've had my computers networked ever since.
Because of where my father worked we were also amongst the first to have a permanent internet connection at home. 24/7 ISDN paid by his employer.
letmeinfornow@reddit
Fun. Was legacy at the time, but I got to briefly work on a 5 mb hard drive with open air platters that were actually platter sized (as in you could have actually serve a turkey on one of them). Buddy kept it in his house. He worked on cat scanners and it came off of one that we retired.
Raphi_55@reddit
you tried to escape and it came back double
BraveBrush8890@reddit
The sad reality is that those that do not understand tech will begin targeting people like us that have home labs. I already had it happen to me, twice.
anomaly256@reddit
They're called neo-luddites by the way. I have a few in n my family of the '5G causes COVID' variety 😓
Alex_1729@reddit
Let me guess, they share Facebook posts filled with Capital letters and share youtube videos asking you to watch as it were evidence?
Also, I thought the 5g conspiracy theory was long obsolete?
SkyFeistyLlama8@reddit
The real 5G conspiracy is carriers blocking older phones or not allowing phones you didn't buy from them from using VoLTE or VoNR.
anomaly256@reddit
They used to - until I blocked them all and went no-contact after their support of DOGE and ICE.
Alex_1729@reddit
Oh, I couldn't do that. I let them believe whatever they want, doesn't affect me, until they start sharing to my closest family. I interject, debunk, and that solves it. I just engaged once or twice, didn't have to any more. They have better things to do that talk about that shit, but I guess there are all kinds of people with nothing better to do.
anomaly256@reddit
I have a young impressionable-aged child they were regurgitating their crap at so I really had no other choice
LegacyRemaster@reddit
me too
FlyingDogCatcher@reddit
yeah... /r/homelab would like a word lol
met_MY_verse@reddit
Especially r/homedatacenter
Andozinoz@reddit
Yeah, this ain't a new thing for many of us. We just got a second GPU this time.
FatheredPuma81@reddit
Huh now that you mention it a friend of mine literally did build a server with a huge overkill server rack, UPS, and so on for datahoarding like 8 years ago... not in the same ballpark since he only spend like $4k on it all I think but still.
texasdude11@reddit
Bahaha, this exactly!
last_llm_standing@reddit
what were you doing with the is the real question
Fit_Assistant7953@reddit
I built a Self‑healing, self‑learning, sovereign AI development engine Runs offline on a laptop or tablet. No cloud, no subscription, forever. Need collaborators to stress test & offer feedback
Model on huggingface. Thanks.
One-Guarantee-2616@reddit
This guy’s an idiot
nvin@reddit
explain
One-Guarantee-2616@reddit
People have been running “data centers” in their homes since the Opteron existed. Now there’s a forum for people to share their experiences which highlights this and the guy thinks its not normal to him? This happens to people who never stepped out of circle in life.
nvin@reddit
I think you are missing the perspective of most people that cannot afford that level of investment into their hobbies.
Massive-Question-550@reddit
well if they knew how big some models were then they would realize that isn't an insane setup at all. or that any server mobo having 256-512gb of ram is pretty typical.
_Erilaz@reddit
It's nothing new. People used to build mining farms much bigger than this.
Heck, 128Gb RAM doesn't sound outlandish at all these days, it's mere 4 times the minimal comfortable number these days, and 64Gb doesn't seem overkill these days. The only reason people started paying attention is the price spike. RAM used to be dirt cheap before MoE models going widespread.
Minute_Attempt3063@reddit
Yeah no, that person has no idea. Has someone explained it properly to then?
IrisColt@reddit
By the way, he's starting to feel that incredibly annoying emotion people call FOMO.
TinFoilHat_69@reddit
I got 4 3090s and 128 gb of ram it ain’t much but it’s honest
IrisColt@reddit
I have only 1x 3090 and 64GB RAM, and I think your setup is neat too.
Affectionate-Pickle0@reddit
Neat. What dock is that?
phido3000@reddit
You don' t know me at all.
But yeh, I was originally like, I will spend $500 on this hobby. It aligns with my work, and my PhD studies. Plus, cool.
But you are literally making things that think, that you can talk to, that can do productive things for you. People spend $5k on a TV or a sound system. Or $50k on a car. But this, this emulates actual thinking.
krzyk@reddit
$500 doesn't cover that amount. 4 GPUs with that size of money is crazy expensive by average people standards. I can't even justify a single 5090 with todays pricing.
Some people here are crazy rich, "I have a 2x H100 in a theeadripper, should I up it with another two?" Or similar.
PeachScary413@reddit
Yeah and it's so depressing seeing someone with that power setup go "So what do I do now guys?? Do I install Ollama or something lol I have no idea"... 💀
krzyk@reddit
Exactly.
I feel a bit outside with my A1000 6GB on laptop and a 3060Ti (8GB) on my remote gaming rig (Moonlight).
phido3000@reddit
$500 didn't cover that amount. That is how much I wanted to originally spend. Setup an old machine with a cheap Mi50..
Then I bought multiple Dual socket xeons, Tb of ram and a palette of Mi50s.. Now I am contemplating re-wiring the house. But everything I bought is now basically worth double what I paid for it. And arguably, it isn't going obsolete as quick as old stuff used to.
My idea or dream is that I can automate much of my work, locally using my own systems. And that I learn about AI in the process.
How much do people spend on getting a degree or for a business.
mohelgamal@reddit
it is funny how people think about these things. if you buy a $50 K boat, that you use a few weekends a summer people would consider this reasonable, but get a $10K computer that you use several hours daily and that feels crazy.
I am even guilty of that, I really struggled to justify to myself buying an $3500 MacBook that I literally use for everything for 3 years nona will continue to use for probably another 2, but it took me a few seconds to justify buying $6000 worth of couches for a living room we rarely sit in.
dangerous_inference@reddit
I just blew a car's worth of money on AI and I have to keep explaining to myself that this is infinitely more important than a car to me. Weird I have to tell this even to myself. We get stuck in outdated value paradigms even after circumstances change.
Sabin_Stargem@reddit
A computer is a car of the mind, and doesn't nearly as much fuel for the freedom you get. If you use a PC nearly all day, it is probably one of the cheapest investments you can make.
Once AI is good enough to work as as effective agents for any digital task, it would be like having a personal lawyer, accountant, entertainer, news host, and educator for $10 bucks a day. A personal AI computer will be just as fundamental as owning a house.
BullableGull@reddit
It's probably not that important to you if you constantly have to force yourself to think that lmao
kmouratidis@reddit
"Buyer's remorse"?
dangerous_inference@reddit
I think I was less pumped about my house being finished.
Minobull@reddit
To be fair I consider the $50,000 boat and $6,000 couches to also be egregious lmao
KellyShepardRepublic@reddit
I had to explain to same to others when they ask me about pcs and MacBooks and then talk about cost. I quickly go to examples like yours and then tell them to stop cheaping out or do, you will use it every day and deal with it anyways.
stoppableDissolution@reddit
Same, yea. "I'll just buy that cheap 3090 to toy around plus its an upgrade anyway"
Two years later its three of them and then the pro 6000 on top lol.
jrg5@reddit
Ledeste@reddit
If he was talking about people with literal datacenter stuff in house, no, it would even be a good wakeup call for some.
But the example is hilarious... Like multi GPU or 128gb of ram was a lot... It more feels like a laptop folk or an envious one that can buy the stuff we want tbh.
ambient_temp_xeno@reddit
Nobody is trying to build a small nuclear reactor to run it, don't worry about it son.
StableLlama@reddit
My laptop had 64 GB RAM before I started with AI, as that's nice to have when running virtual machines.
What's the issue with 128 GB?
Ok_Try_877@reddit
Bro.. you stick with your GF's, fake tan and and peptide injections... and I'll stick with my 512GB RAM and way more cards.... 128GB? You must be talking about my laptop and client PC's?
New-Implement-5979@reddit
Not at all
Sabin_Stargem@reddit
I hope we are building personally owned data centers. IMO, corporations shouldn't own compute nor robots - they should be renting those things from individual households. This would make it much harder for corporations to do evil and stupid things, if they have to answer to actual people.
Democracy, requires many individuals to have a slice of power and no one controlling a large part of the pie.
PeachScary413@reddit
With current prices it makes absolutely zero sense lmaoo
But it's a hobby for rich people and it's fun to build stuff so :)
audioen@reddit
LLMs perform valuable work, and by a projection there is a payback period for an investment in LLM. For Western frontier model, I would probably use much more than $10 per day, and for a Chinese model, around $5 per day. Depending on the prices and quantity of mechanical intellectual labor you use, you'll probably discover that you're doing more than $1000 worth of AI compute each year. To me, it's pretty much a nobrainer that you'd put in little money to get lots of value back.
Back in the day when you could get the ASUS GB10 for like $2000-3000 price, it was relatively cheap considering the value. Hell, I should have bought two, probably. I'd be running 256 GB range models with decent enough compute and bandwidth today. I regret not doing it now, but I had just bought Strix Halo box earlier that year with 128 GB and I felt weird about buying yet another 128 GB box for more AI stuff.
The real stuff for multiuser remains very costly -- $100k and up is in the ballpark for serving an organization, though with dozens to few hundred users, the math fundamentally doesn't change very much. We can probably bring the insane compute requirement in prefill down in future models, and turn the sequential inference into more like block inference with approaches such as dflash. This way, the models better match the limitations of common hardware, and these consumer boxes many of us have today become more useful.
SolarNexxus@reddit
You have to pump up thos rookie numbers. Home server starts at 1024gb of vram.
FragmentedHeap@reddit
.........
FragmentedHeap@reddit
9950 x3d with 196 gb ddr5, a 4090, and a 6950xt....
Gringe8@reddit
Hey i have the same case i think and cpu. Only 96gb ram, but with a 5090/4080
FragmentedHeap@reddit
it is a nice case, I have my 4090 vertical with the riser cable, mounted in the side aio tray, aio is on top of case. 6950xt in main slot.
Once new motherboard goes in I'm a get a 2nd 4090 prolly eventually
FragmentedHeap@reddit
New motherboard going in tmrw so I can run dual 4090s at 8x+8x.
VoiceApprehensive893@reddit
buying gpu's is better than renting
ea_man@reddit
Tell the guy I got an old Sun Ultra workstation worth some 20K (at the time) as a door stopper.
don't ask what I was running by the time of MOSIX whas the cool thing.
CalligrapherFar7833@reddit
Hows your pension fund looking old man
Long_comment_san@reddit
2x3090 +128 gigs of RAM will do 95% of the tasks.
Librarian-Rare@reddit
Anyone who doesn’t have AT LEAST a few Nvidia DGX B200 servers to run qwen 3.6 35b a3b; then cmon, are you serious.🧐
datbackup@reddit
Anyone who isn’t wearing 2,000,000 SPF sunblock
Metalmaxm@reddit
Why do you care?
Are you not a man? You don't have your own spine?
Grow up, be a Man.
If this is your hobby, like me. It their problem, not my. It is my money, time, life, not their.
You don't need such toxic people in your circle, if they talk, envy about your hobby.
tecneeq@reddit
If you don't run a local AI datacenter, you are falling behind, bro.
BangkokPadang@reddit
DDR5 VRAM sounds terrible.
miniocz@reddit
Why? My p40 runs fine.
thisguynextdoor@reddit
That's true, even if we don't have powerful rigs. The compute and inference happen always somewhere. It can be your phone, your laptop, or a 4x RTX rig.
It's just that datacenters are far more energy-efficient than any home or office setup.
heliosythic@reddit
Do people not realize the issue with "datacenters" isn't that its a datacenter existing, its that its loud and over-uses local and rare global resources and provides virtually no ongoing jobs. There'd be no issue if they didn't effect others, the point is it does and they dont care because it makes them money at our expense. Having your own little home datacenter paying your own electric bill and having to hear fans in your own house that doesn't effect anyone else is a totally different thing.
ChopSueyYumm@reddit
In the cryptocurrency goldrush people had a home crypto mining data center in their garage.
huldress@reddit
Idk, when you're the average person looking to get into LLM, DetectiveNarrow's post sums up exactly how it feels trying to get into it lol
It feels hopelessly lost and the people telling you how wonderful local is are also the ones with the hardware to actually be able to enjoy it. Meanwhile, you'll realistically end up trying to run a much smaller model and if you've ever tried LLMs before it won't be as enjoyable as the API models.
Of course, usebase will have this opinion vary greatly. Point still stands.
eggs-benedryl@reddit
Truly, I treated myself to a 5090 PC and can run what I want to run but even then some of the coin people are dropping for this stuff is wild. I gets a hobby but damn
an0maly33@reddit
I have machines that I use for gaming, work, and development anyway. Using them for AI here and there is a marginal difference.
Double_Ad9821@reddit
Nothing wrong with that. We need to end the monopoly in the name of efficiency
Billthegifter@reddit
I mean It Is true but at the same time people are forgetting the difference of scale. Sure If I have 3 5090's I'm going to be using a lot of power but It's going to be basically nothing when compared to even one of the racks In a data centre
05032-MendicantBias@reddit
Wait until OOP learns about 3D printing!
Altruistic_Heat_9531@reddit
Before LLM, i have actual data lake on my home
LetsGoBrandon4256@reddit
Dude sounds like he just discovered what a hobby/mancave is.
Agitated-Fly3564@reddit
He must be glad
InevitableLimit6020@reddit
Mine doesn't suck an entire municipal water supply dry.
fugogugo@reddit
damn I feel so poor reading all these comments
Bulky-Priority6824@reddit
My buddy said nerds would come out of the wood works to create/use a super genius friend- that's why the AI explosion
LankyGuitar6528@reddit
It's OK. I bought a flat of water at Costco for my neighbors to make up for all the water I'm using in my data center.
Witty_Mycologist_995@reddit
The can it run doom of localllama: can it run qwen
MathmoKiwi@reddit
"4 GPUs" = a data center
lol, he has no clue whatsoever
DreamingInManhattan@reddit
A datacenter in my home was my intention all along. Couldn't be happier. You can't believe how much you can get done with a small army of qwen 35b agents.
Dry_Yam_4597@reddit
That is pathetic tbh.
dago_mcj@reddit
I'm slightly concerned the redditor might have read my post earlier. https://www.reddit.com/r/LocalLLM/comments/1tpsdw7/my_home_lab_7900x_dual_x16_build_w_rtx_3090_and/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button
Equal_Giraffe8866@reddit
bitch please i have a datacenter hooked up to a 200W galvo laser please come correct with your awe
sloptimizer@reddit
I have only one response to this person:
Ummite69@reddit
2x5090, 1x3090, 2 24 core pc with total 352 gb ram ddr5, 250tb total san space 10gbit network... Will purchase rtx6000 pro end of summer. Just a hobby, but I'm a programmer (still) so I use it for multiple purpose including not paying for a claude with qwen3.6 27b Q8
norman_h@reddit
lolz. I've got 18 GPUs, about 3tb of ddr4 and half a petabyte of storage... all running off a 32 amp circuit... yes, it's a tier three data centre.
Key_Solid_1696@reddit
They look at me weird when I explain my 'data center' uses, on average, less than the power of a 60W light bulb (M3 Max with 128GB) each day. I am currently running DS4 Flash Q2 with 768K of context on it. Not fast, but much faster than my brain for what I use it for (coding assistant, security reviews, etc).
crawler00000@reddit
he?
... yeah, likely he
honestduane@reddit
I mean, I do have a full server rack in my server room with the network hard link, but that's normal, right?
suesing@reddit
Who cares what other people’s opinions are. They’ll never understand the joy you get from your hobby. So live your best life
letmeinfornow@reddit
I only have 3 gpus in my AI rig....I feel so inadequate. In contract I have 256GB Ram, so there is that. 😉
scarbunkle@reddit
Honestly, most of my spend was pre-AI and continues to be non-AI hosting stuff.
And yeah, my rack server has some brilliant industrial design. Should have bought one sooner.
Caramel-Secure@reddit
Well… the word “detective” is in the posters name.
Well… so is “narrow”, so who knows!
dangerous_inference@reddit
I kill firstborns while their mamas watch. I turn cities into salt. I even, when I feel like it, ^(run 4x 48gb modded 4090s), and from now till kingdom come, the only thing you can count on in your existence is never understanding why.
Pleasant-Shallot-707@reddit
Yes
a_beautiful_rhind@reddit
Yes, and for cut rates.
PwanaZana@reddit
it's like saying you garage is a industrial warehouse, no, one workstation is not a data center, it's not even a data edge or data corner.
Slasher1738@reddit
First off, they should mind their business
NeedsSomeSnare@reddit
Their post is in response to people talking publicly online. That's not a "mins your own business situation.".
You're also assuming they're criticizing you, rather than just pointing something out. Don't be so easily triggered.
Slasher1738@reddit
Outta here with that. Community discussion supporting the community. I get annoyed with people that think they're the neighborhood watch.
99% of people are running something so low power it wouldn't register on the grid. Everything we're doing combined pales in comparison to 1RU in an a AI datacenter
NeedsSomeSnare@reddit
You're the one acting like neighbourhood watch though... Their comment section seems like a very friendly discussion.
Also, its fine to criticise a community when needed.
Bulky-Priority6824@reddit
pretty triggered response i should say lol but i agree
Bulky-Priority6824@reddit
man people been doing this for decades, its only now its something universally interesting unlike overclocking or mining or even gaming which most people grow out off.