PS5’s can now be hacked to run Linux - perhaps some potential for local inference?
Posted by Thrumpwart@reddit | LocalLLaMA | View on Reddit | 74 comments
I look forward to the Local LLM community getting llama.cpp to run on these. Could be a good value.
ArthurOnCode@reddit
AMD Radeon GPU based on the RDNA 2 architecture, 16 GB GDDR6, 448 GB/s. Sounds workable.
Eyelbee@reddit
Absolutely, and these are probably cheaper than the cheapest 16GB rig you can buy today. Too bad 16GB isn't any good for most tasks. If they were 24GB it could actually make sense to get one and turn it into a server.
fallingdowndizzyvr@reddit
A BC250, even at today's elevated prices, is a fraction of the cost. And is pretty much the same.
TheManicProgrammer@reddit
Could you elaborate please?
fallingdowndizzyvr@reddit
The BC250 is made from rejected processors for the PS5. They generally are missing a couple of cores but are otherwise the same spec as a PS5. It was a SBC made for mining. They used to be dirt cheap, like 12 for $350 so about $30 each. Now they sell for about $150.
TheManicProgrammer@reddit
Two of these seem like a pretty good approach
fallingdowndizzyvr@reddit
I don't think so. When they were $50, sure if you want to blow $100 to play with them why not? But they are $150 now. So two would be $300. For $300 you can get more with less hassle. $300 would buy you two V340s and a cheap PC to put them in. Then you would have more VRAM, potentially much more speed(through TP) and much less hassle.
TheManicProgrammer@reddit
V340 looks a lot more unstable from the few forum posts I've seen? Cost wise you have a point, but I'm not US based so local markets don't even have a v340 here haha
fallingdowndizzyvr@reddit
Than a BC250? No way. Since the V340 is pretty stable. It just plug and play on Linux. Use Vulkan to make it easy. Now it's not going to be fast. Neither is the BC250. But unlike the BC250, the V340 has potential. That being TP. Which you can do even with one card.
Miller4103@reddit
They would probably cluster them.
yeah-ok@reddit
That was the original premise with the the PS3 ... https://phys.org/news/2010-12-air-playstation-3s-supercomputer.html
mikkolukas@reddit
I remember that. It was so cool back then
yeah-ok@reddit
Yup, I was reading Kurzweil's "The Singularity Is Near" book back then and feeling the techno-end-times vibe
FreedomByFire@reddit
You should take a look at the BC250 is essentially a cut-down PS5 APU that you can often find on eBay for under $100. There is a decent community around it, and some people use it for local inference because it has high-bandwidth memory. The main limitation is that it only has 16 GB of RAM, which can be a problem depending on the model and workload. This would be the same issue for PS5, and my opinion would not be worth buying at the current price.
fallingdowndizzyvr@reddit
It doesn't even have that. Since you can't allocate all of it to the GPU. I think the max is like 12GB.
FastDecode1@reddit
Workable maybe, but not very good.
RDNA 2 has no matrix acceleration whatsoever, for any sort of AI shit you'd want at least RDNA 3.
legit_split_@reddit
As always it just depends if the price is right
Eyelbee@reddit
I was getting great speeds with my 6800 xt, the bottleneck is the bandwidth
Thrumpwart@reddit (OP)
PS5 Pro has rdna3, but I don’t know if the Linux hack works with pro.
No-Refrigerator-1672@reddit
Look up AMD BC-250. It is a mining blade that uses PS5 APU, with 2 less cores and 20% less GPU performance, if I remember correctly. It costs way less that PS5 while being essentially the same thing, so if you want to toy around with it, you can do it without sacrificing your console.
fallingdowndizzyvr@reddit
Back in the day, you could buy a whole server with 12 BC250s in them for $350. Not $350 for each BC250, but the whole server with 12 BC250s for $350. So each BC250 was about $30. People bought them and broke them apart to sell each BC250 for $50.
No-Refrigerator-1672@reddit
Yeah, the prices for anything that can compute large matrices is skyrocketing right now. Well, what can you do. It still costs like $250-$300, with PS5 bieng at least twice as expensive.
fallingdowndizzyvr@reddit
A BC250? No. Even with today's elevated prices, they are like $150.
No-Refrigerator-1672@reddit
Yeah, if you're in the US. In the other world - nope.
fallingdowndizzyvr@reddit
Right here on German Ebay. 123 EUD AKA 145 USD. So $150.
https://www.ebay.de/itm/287227410833
k1rika@reddit
Well yes but also no: "US $541,83 (ca. EUR 464,02)" is what it shows me on that very page for corresponding shipping costs. So you are right about the unit price but if you don't buy large amounts of them it's still way more then the ~$150 buying from GER for instance.
fallingdowndizzyvr@reddit
I don't see how shipping would be $400 even accounting for VAT. But anyways buy it for ~$150 from GER than. Like that one that sold on April 6 for 119 euro located in Germany.
k1rika@reddit
Don't ask me why it is that expensive, but it really is. Maybe taxes, unfortunate conditions by parcel service ("UPS Worldwide Expedited^(SM")) Logistic Issues with the current geopolitical situation or something like that:
FreedomByFire@reddit
i bought a brand new bc250 for $70 a few months ago. They're no where near 350.
fallingdowndizzyvr@reddit
Perhaps you can run a LLM on it to summarize my post. Since you are missing the point. Here, let me quote it for you.
"Not $350 for each BC250, but the whole server with 12 BC250s for $350. So each BC250 was about $30."
Thrumpwart@reddit (OP)
Does it come with a controller through?
YairHairNow@reddit
Seems like you wouldn't be able to play games if you're on a 4 year old firmware. I've been thinking about getting a console for GTA6 and this would push me to get one but unless I can do both. Not interested.
-dysangel-@reddit
If you already happen to have one or want one for gaming, this could be a nice excuse to push you over the edge, but with only 16GB of VRAM and 2GB of system RAM, I definitely wouldn't buy one just for inference.
CarelessOrdinary5480@reddit
2gb? That's rough, I assume you could use disk cache to load the model into the vram?
RoyalCities@reddit
The hackable firmware is from basically 2021. Chances are low you'll even find one that can use the exploit.
miklosp@reddit
“ Users who want to downgrade or sideload a specific firmware version can do so using Sony's official reinstall process with the correct PUP file”
CCloak@reddit
Only true for PSP, and even then, partially true (Pandora Battery was an exploit in older PSPs that can trick the hardware to boot FW from Memory stick instead, which would then let you reinstall almost any FW version into the console.)
RoyalCities@reddit
This is incorrect. Sony's hardware only allows forward only firmware flashes. If you're on say firmware 5 it'll reject a firmware 3 or 4 etc.
tm_142@reddit
I dont think thats correct. Otherwise no one would stay on the lowest firmware as possible in hope they can jailbreak someday
ValenciaTangerine@reddit
yeah agreed, 16GB GDDR6 at 448GB/s puts it roughly in 3060 Ti territory for inference, which is fine for q4 27B but nothing earth shattering. the AMD BC-250 mining blade someone else mentioned is the cheaper version of the same APU and you can get them on ebay for around $200. the PS5 angle only really makes sense if you already own one and want a side project, the math doesnt work as a primary inference rig vs just buying a used 3090.
dev1lm4n@reddit
3.7 times higher memory bandwidth than M4 Mac Mini though
Dany0@reddit
You used to be able to connect Ps3s with some high speed connection, that's how they built that supercomputer
But I think that was as ibm specific tech
AmericanNewt8@reddit
Yeah Cell was wacky for a whole host of reasons. Consoles have mostly gone to largely off the shelf x86/arm chips though.
Bootes-sphere@reddit
Interesting idea! The PS5's GPU is decent for inference, but you'd hit some real constraints. The 448GB/s bandwidth sounds great until you realize it's split between CPU/GPU, and you'd need to optimize heavily for the custom architecture. Smaller models (3-7B range) would work better than larger ones.
Fit_Advice8967@reddit
With the right small model with the perfect amount of quantization i would turn my ps5 i never use into my ai homeserver
LinkSea8324@reddit
Fun project but useless
delveccio@reddit
I remember back when you didn’t have to hack a PlayStation to run Linux on it
Lorian0x7@reddit
yeah, In the days when we owned what we bought.
ItsAMeUsernamio@reddit
If the next Xbox (Project Helix) has UDNA and can run Windows with atleast 30GB GDDR7 as RAM, that’s going to be a hit on here.
MrMPFR@reddit
Based on the memory controller it's most likely 36GB GDDR7.
10-8GB for background, CPU and OS leaves plenty for VRAM.. Or +30GB for LLM tasks.
Will be interesting to see how impressive the ML HW will be.
Noiselexer@reddit
Ofc not
nmrk@reddit
I saw that episode of POI.
hallofgamer@reddit
Here comes the openclaw ps5
send-moobs-pls@reddit
Anthropic paper: We Told Claude To Act Like A Call of Duty Player and You Will NOT Believe What He Said!!
andy_potato@reddit
Probably something about how deep his relationship is with your mom.
More-Curious816@reddit
FreedomByFire@reddit
You should take a look at the BC250 is essentially a cut-down PS5 APU that you can often find on eBay for under $100. There is a decent community around it, and some people use it for local inference because it has high-bandwidth memory. The main limitation is that it only has 16 GB of RAM, which can be a problem depending on the model and workload. This would be the same issue for PS5, and my opinion would not be worth buying at the current price.
Strange-House206@reddit
The bc-250 is basically this and way easier to set up. Not really worth it for that purpose tbh
fallingdowndizzyvr@reddit
Just go look up a thread on the BC250. Which is pretty much what AMD did with the leftovers from building PS5s.
Considering that even when BC250s were $50 each it wasn't, I don't think a PS5 would be either.
VoiceApprehensive893@reddit
its a 16gb gddr6 rig essentially could run q3.6 27b
bonobomaster@reddit
This is a very strange and unrealistic assumption.
Even with 24 GB VRAM you can only use Q5 and lower quants, if you want to have any context at all.
Monkeylashes@reddit
hardly enough to be useful, even with q4 you'd hardly have any usable room for context left, and quantizing the kv cache butchers the output quality.
ExplodingFistz@reddit
Might be worth looking into this then
_angh_@reddit
You will probably buy cheaper mac m1 mini and get much better results oob.
inaem@reddit
After what happened with Mac mini I guess we are not going to be seeing PS5 anymore
DarkArtsMastery@reddit
yeah no
Thrumpwart@reddit (OP)
No, yeah!
elgafas@reddit
Chappie was able to build this in a cave! With a bunch of PS4s!
JapaneseBidetNozzle@reddit
Enough. You don’t have to use every computational source for inference or crypto mining or your nonsense hobby use.
You are just contributing inflation of hardware prices, I don’t even think most people creates value from the hardware they own just because of a stupid hype.
wind_dude@reddit
back in 2010 the US airforce connected 1760 PS# to process satellite imagery, https://phys.org/news/2010-12-air-playstation-3s-supercomputer.html
myreala@reddit
Can't really connect them together with a 1Gpbs Ethernet port. Even the USB-c is just 10Gbps. Not sure if that's enough.
ImaginationKind9220@reddit
PS5 is around the speed of a 3060, it was fast in 2020 but that's 6 years ago.
AshuraBaron@reddit
Specs aren't that impressive. GPU is decent, but it's not more powerful that what you could find from any mini PC's these days. Total cost of ownership might be cheaper, but not by much. It's a cool application for it, but I don't foresee a rush on PS5s to daisy chain together. Especially with the price increases.
volca02@reddit
Just get bc-250 if you want to experiment on a similar hardware. Kind of hard to get outside of US though.