4 x 5070 Ti dual slot in one build?

Posted by Mr_Moonsilver@reddit | LocalLLaMA | View on Reddit | 12 comments

I was looking at the Inno3d X3 RTX 5070 Ti two slot design, and wondering if it would be possible to stack 4 of them into one well ventilated case and if temps are going to be ok? I'm not looking to run models in tensor parallel, running individual models on each card like ASR, embedding, rerankers and OCR pipeline, so not all are running at the same time constantly.

Reply to Post

12 Comments

[-]

mrsavage1@reddit

u/Mr_Moonsilver did you actually build this in the end?

[-]

Mr_Moonsilver@reddit (OP)

No, I didn't because prices went up 30% on those cards. Still think it's a good build

[-]

Similar_Idea_2836@reddit

Hi, OP do you mind sharing the use of 4 x 5070 Ti setup in LLM inference or other AI applications you have tried on it ? I also think about having the same setup with ASUS series.

[-]

see_spot_ruminate@reddit

I don't know that the 5070ti offers significant benefits unless you are doing something like gaming. If you are just going for an amount of vram with 4 cards, then look into the 5060ti. If you want something more beefy a 5090 might be more worth it.

[-]

LittleFollowing1190@reddit

Eh depends on your use case though - if you're running separate workloads like OP said then having 4 separate cards actually makes sense for isolation, plus you get way more total VRAM than a single 5090

[-]

Mr_Moonsilver@reddit (OP)

Thank you for the reply. I want a decently fast GPU to run different, smaller models. The 5090 would not only be twice as expensive for the same amount of VRAM but require me to run two models on the same card. I need dedicated cards for dedicated models.

[-]

see_spot_ruminate@reddit

I don't know that its going to be that much faster for the money, but it is your money.

[-]

StorageHungry8380@reddit

I had a 5070 Ti which, due to reasons, I had side-mounted in my Fractal Define 7 case. I had the Asus TUF card with its huge cooler, so the fans had just a 3mm gap from the side panel soundproofing. I gamed without issues in that configuration, in games that pushed the card to its power-limit, and temps were just fine. During inference workloads I typically saw much lower power draw, more like 200-250W. I think I'd be fine with that setup, assuming you have adequate case fans to provide decent ambient air temps inside the case.

[-]

Mr_Moonsilver@reddit (OP)

Thank you for a very useful comment indeed! This gives me confidence to go ahead and try it. I can still space them if it doesn't work! Thanks a lot for a comment with real data.

[-]

ThunderousHazard@reddit

Given your requirements, honestly two 5060TI 16GB could suffice with quantized models. That said, since you have the budget, do you also have the space for an open mining rig? You could easily create an inference server with 4x5060TI 16GB running out of a single 1KW PSU (power limit to 150W, performance doesn't decay much if at all). This does give you the freedom to upgrade easier later if need be (depends on the PCIE slots on the motherboard though) as you could overspec the PSU by grabbing a 1.5KW one.

[-]

Mr_Moonsilver@reddit (OP)

Thank you for the reply, it needs to go in a rack too ;) But that's not an issue, I have 3xT30s on full speed supplying enough air for cooling. and I would prefer to have the 5070 TI for the higher bandwith. It's for the company I work at and it's going to support the chatbot with embeddings and reranker for around 30 people. It could work probably with the 5060 TI but since I have the budget for the 5070 TI I thought I'd rather want to be on the safe side. The power limit makes actually a lot of sense. I think with that it will work just fine.

[-]

_hypochonder_@reddit

It what be loud and hot. I used a AMD RX 7600XT 2 slot behind a other card. (PowerColor Fighter) It run really hot and I put the fan on 100% and lower the TDP. RTX 5070ti has 300W as TDP. In the end you would have 1200W in your case when you use vLLM as worst case. I would use the the worst case because you can use it like furmark or prime95. Use a miner case with PCIe riser cables.