Need help for a machine Im building
Posted by spybehindyou99@reddit | LocalLLaMA | View on Reddit | 6 comments
I bought a supermicro x10dri-t4 and 2 xeon e5 2699 and 64gb ram (4/16 slots so far) all used on ebay and very cheap, but now to keep this arround 1500€ I cant decide between 4 tesla p40s from FB marketplace and 4 chinese modded 2080ti 22gb from alibaba.
Yeah. Outrageous. But I cant see any other budget option to get loads of vram. Everything is shady, or way out of my league. I dont know if its more likely I will get bent over by some stranger on marketplace or by a chinese company in china where atleast there are things like trade assurance and order monitoring services.
the p40s would run me about 800€, the 2080s are 1200 with protections and shipping
MelodicRecognition7@reddit
if there are enough PCIe lanes & slots in 1 CPU for 4 GPUs in 1 CPU then you should remove the 2nd CPU to avoid NUMA issues.
spybehindyou99@reddit (OP)
thanks. I jumped the gun a bit when I saw this board for under 100€. unfortunately with 4 gpus im not going to have any other choice other than to deal with it I dont think… what do you think of the gpus?
MelodicRecognition7@reddit
both are too old and not supported by current software libraries, V100 32GB is the oldest I'd use, but due to the world's AI psychosis they became prohibitely expensive.
spybehindyou99@reddit (OP)
So there are no options? The 3090 pulls 350-400 watts for the same vram, the 4090 even more. the v100 right now is as shady as the modded gpus and as expensive as the 3090s… The original plan was vram for as cheap as I could get it but for 50-70 more per gpu I could get 8 less vram and slightly faster inference. the v100 and 3090 get like 50% more tokens for double the price and 50% more power draw… Is this a hobby only for the rich now?
MelodicRecognition7@reddit
well P40 is still an option but with caveats, search this sub for "p40". 3090 is still the gold standard, it does not just "pulls more watts", it has 3x memory speed and Nx performance compared to P40.
spybehindyou99@reddit (OP)
The 3090 has 3x the speed of a p40, 4x the cost, and almost 2x the power draw. When you start comparing to the “hacked” chinese 2080tis the numbers get even bleaker. And I have 2 amazing deals for 3090s for sale on facebook marketplace for under 1200€ total, but I feel like thats the cap, no scalability, because I cant afford to have a space heater that costs me 50-70€ a month or want to have a server PSU or multiple PSUs, I probably would need to redo the electricity at my house… And I dont think 48gb of vram is enough to play arround with. But clearly I havent made my mind up yet, and I dont know whats riskier, a random seller on facebook that can block me tomorrow or a alibaba GPU. Thank you for the replies, I am knees deep at this point, and feedback/help is invaluable.