Pics of new rig! | TheaterFire

Pics of new rig!

Posted by WyattTheSkid@reddit | LocalLLaMA | View on Reddit | 54 comments

Just recently redid my workstation rig in a Phanteks enthoo pro 2 server edition case with 2x 3090 TI FE and 2 3090s. Thanks to u/faileon for the inspiration and reference build. I am going to install more fans but its almost 5AM for me and im a little buzzed ngl but I just wanted to share my progress because I just finished it and made sure it posts and im really excited about it because I worked my ass off and saved a shit ton of money to pull this off. Very proud of my monster. :)

Goodnight everybody!

Oh and the temps are actually pretty good, the way I have the cards mounted seems to work well for airflow.

[-]

Powerful_Evening5495@reddit

I am in love. Let's get married and vibe code together for the rest of our lives.

[-]

a201905@reddit

That made me laugh. Love this community

[-]

WyattTheSkid@reddit (OP)

Isn’t she beautiful? 😍

[-]

StupidityCanFly@reddit

I like the Phanteks case. Mine has 16 fans and dual rx7900xtx inside. Need to figure out if I want to add two more 7900s or go quad ai pro 9700 route.

[-]

legit_split_@reddit

How do you like the 7900xtxs, have you tried the latest tensor parallelism implementation in llama.cpp?

[-]

StupidityCanFly@reddit

I haven’t tried it yet. Right now it’s running vLLM. I need to look at llama.cpp’s performance next week.

[-]

legit_split_@reddit

I see, have you been running Qwen 27B? I'd love to see some benchmarks.

[-]

StupidityCanFly@reddit

Just run a quickvllm bench on Qwen3.6-27B.

Results:

============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Maximum request concurrency:             4
Benchmark duration (s):                  448.50
Total input tokens:                      25128
Total generated tokens:                  22749
Request throughput (req/s):              0.22
Output token throughput (tok/s):         50.72
Peak output token throughput (tok/s):    48.00
Peak concurrent requests:                7.00
Total token throughput (tok/s):          106.75
---------------Time to First Token----------------
Mean TTFT (ms):                          2481.01
Median TTFT (ms):                        2158.72
P99 TTFT (ms):                           9158.45
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          67.27
Median TPOT (ms):                        54.66
P99 TPOT (ms):                           407.79
---------------Inter-token Latency----------------
Mean ITL (ms):                           159.05
Median ITL (ms):                         102.38
P99 ITL (ms):                            2775.11
---------------Speculative Decoding---------------
Acceptance rate (%):                     71.91
Acceptance length:                       2.44
Drafts:                                  9326
Draft tokens:                            18652
Accepted tokens:                         13413
Per-position acceptance (%):
  Position 0:                            81.13
  Position 1:                            62.70
==================================================

More testing after some sleep.

[-]

StupidityCanFly@reddit

So, that took a while, but after some tuning, vLLM showed these numbers below - any concurrency above 4 and it slowed down.

============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Maximum request concurrency:             4
Benchmark duration (s):                  316.77
Total input tokens:                      25128
Total generated tokens:                  22749
Request throughput (req/s):              0.32
Output token throughput (tok/s):         71.81
Peak output token throughput (tok/s):    48.00
Peak concurrent requests:                6.00
Total token throughput (tok/s):          151.14
---------------Time to First Token----------------
Mean TTFT (ms):                          801.74
Median TTFT (ms):                        501.17
P99 TTFT (ms):                           5456.07
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          48.88
Median TPOT (ms):                        47.57
P99 TPOT (ms):                           84.78
---------------Inter-token Latency----------------
Mean ITL (ms):                           118.62
Median ITL (ms):                         110.45
P99 ITL (ms):                            366.86
---------------Speculative Decoding---------------
Acceptance rate (%):                     70.55
Acceptance length:                       2.41
Drafts:                                  9431
Draft tokens:                            18862
Accepted tokens:                         13308
Per-position acceptance (%):
  Position 0:                            80.04
  Position 1:                            61.06
==================================================

But what's interesting, I also tested llama.cpp (main as of yesterday). It was running best at 16 concurrent requests.

============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Maximum request concurrency:             16
Benchmark duration (s):                  319.28
Total input tokens:                      25128
Total generated tokens:                  22749
Request throughput (req/s):              0.31
Output token throughput (tok/s):         71.25
Peak output token throughput (tok/s):    125.00
Peak concurrent requests:                18.00
Total token throughput (tok/s):          149.95
---------------Time to First Token----------------
Mean TTFT (ms):                          2123.53
Median TTFT (ms):                        1637.89
P99 TTFT (ms):                           5562.71
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          204.28
Median TPOT (ms):                        206.05
P99 TPOT (ms):                           330.18
---------------Inter-token Latency----------------
Mean ITL (ms):                           198.54
Median ITL (ms):                         140.72
P99 ITL (ms):                            1535.88
==================================================

HW Specs:

CPU: Intel Core Ultra 5 245KF
Mem: 64GB
+------------------------------------------------------------------------------+
| AMD-SMI 26.2.2+671d39a71e    amdgpu version: 6.16.13  ROCm version: 7.2.2    |
| VBIOS version: 00088704                                                      |
| Platform: Linux Baremetal                                                    |
|-------------------------------------+----------------------------------------|
| BDF                        GPU-Name | Mem-Uti   Temp   UEC       Power-Usage |
| GPU  HIP-ID  OAM-ID  Partition-Mode | GFX-Uti    Fan               Mem-Usage |
|=====================================+========================================|
| 0000:04:00.0     Radeon RX 7900 XTX | 0 %      43 °C   0            19/294 W |
|   0       0     N/A             N/A | 1 %      0.0 %          18642/24560 MB |
|-------------------------------------+----------------------------------------|
| 0000:07:00.0     Radeon RX 7900 XTX | 0 %      40 °C   0            19/294 W |
|   1       1     N/A             N/A | 1 %      0.0 %          16582/24560 MB |
+-------------------------------------+----------------------------------------+

[-]

Geximus-therealone@reddit

Better to use custom case with load fans

[-]

WyattTheSkid@reddit (OP)

That things nuts what kind of case is that? Also what kind of power supply even exists that can power that thing lmao

[-]

Geximus-therealone@reddit

113090 2leadex flower titanium 2200w. But I use rtx3090 with down voltage 180w (if I need more ts I use 230w +15% ts). Custom case crypton for 12gpu.

[-]

BenniB99@reddit

Damn

[-]

Thrumpwart@reddit

Very nice, and kudos for saving up and making it happen. A labour of love like this is so much more rewarding than a corporate black box that costs 4 times as much.

[-]

Tritheone69@reddit

Would you mind sharing your motherboard? I’m currently wondering if it’d be worth it to attempts a 2x RTX3090 + 2x RTX3080 build. My main concern is the amount of PCIe lanes I’ll be sending to each GPU.

[-]

cleversmoke@reddit

Beautiful rig! What you thinking of running on them?

[-]

WyattTheSkid@reddit (OP)

Minimax m2.7 ☺️

[-]

cleversmoke@reddit

The dream! 🤩

[-]

ixtrqc@reddit

Beautifoul but i doubdt the air circulation will be optimal, even with other fans. Should consider an open frame casse IMO

[-]

WyattTheSkid@reddit (OP)

It needs to fit under my desk unfortunately. I might leave the side panel off tho

[-]

Cferra@reddit

Time to bit the bullet and water cool those cards and do it right.

[-]

WyattTheSkid@reddit (OP)

I plan to do that when I have the money but have you seen the prices of waterblocks for these cards? They’re insane

[-]

Cferra@reddit

Around like 150 I think. That’s what I got mine for

[-]

WyattTheSkid@reddit (OP)

I could find any for less than like 250 a piece which would mean another 1k~ to water cool the thing. Definitely in my plans at some point but before I do that im gonna get it integrated with a proper workstation board and a threadripper. Waiting for ram prices to stabilize first though

[-]

Cferra@reddit

I’ve got 2x plus nvlink on a c422 sage10g board along with 2x b50 pros split it up with proxmox and then another server with the same board with 4x 5060 ti 16GBs

[-]

WyattTheSkid@reddit (OP)

Thats badass

[-]

Adventurous-Paper566@reddit

Comment faites-vous tenir les GPU verticalement de la sorte?

[-]

WyattTheSkid@reddit (OP)

I have 2 separate brackets, one for the gpu in the back and one for the gpu in the front. The one in the back is screwed into the top of the case and the one in the front is screwed into the bottom side part of the case

[-]

brickout@reddit

Wt actual f

[-]

WyattTheSkid@reddit (OP)

Thats what I said the first time it powered on

[-]

BitGreen1270@reddit

Man that looks stunning, 4 graphics cards! So jelly. If you don't mind sharing, what was the ballpark amount for this? I'm guessing this is ~8-9k usd?

[-]

WyattTheSkid@reddit (OP)

I don’t know for sure honestly but based on the current used market you could probably build something very similar for about 9k yeah. I know I paid 2800something for all 4 gpus on fb marketplace over the span of a year. I can send you the whole part list if you want

[-]

BitGreen1270@reddit

Thanks for sharing - nah I will chug along on my potato laptop for now :). Can't afford it :D.

[-]

WyattTheSkid@reddit (OP)

I can’t really afford it either tbh lmao. I bought the parts slowly over almost 2 years

[-]

Kerem-6030@reddit

😍😍😍

[-]

cutter89locater@reddit

Would mind to share about the hanging GPU? I see the riser cable there, but how to secure it further? I'm gonna do the same thing for my 3rd card, cos every bit of vram counts XD

[-]

WyattTheSkid@reddit (OP)

Which one, the TI in the front or the EVGA one in the back?

[-]

WyattTheSkid@reddit (OP)

Which one, the TI in the front or the EVGA one in the back? If you dm me i would be happy to send pictures of how they’re mounted and send amazon links to the exact brackets and riser I used

[-]

JockY@reddit

Bro rocking the Nickelback and Foreigner posters!

[-]

WyattTheSkid@reddit (OP)

Hell yeah brother 😎

[-]

Fearless_Ad_1045@reddit

what are the specs?

[-]

WyattTheSkid@reddit (OP)

Ryzen 9 5950X w/ Scythe Ninja 5 cooler

128gb of DDR4 @3600mhz (2x 64gb g.skill ripjaws kits)

Asus ROG STRIX X570-E WIFI II mobo

2x RTX 3090 TI FE + 1x Gigabyte RTX 3090 + 1x EVGA 3090

For mass storeage: 4tb Seagate barracuda pro + 12tb seagate barracuda + 20tb random western digital oem drive I found on eBay. For my OS and apps that do a lot disk operations I have a western digital sn570 2tb lil m.2 ssd

Thermaltake Tf3 1650w titanium psu (this fuckng thing cost me 475$ brand new and I had to buy it because the psu I got on ebay used didn’t work and I needed to buy a good one that amazon prime would get to me fast :/. Lowkey worth it tho because it will inevitably save me money too off the electric bill when I do PEFT and training on this thing.

Case is a Phanteks enthoo pro 2 server edition as mentioned in the top of post I know people don’t usually mention fans in a spec sheet but the phanteks t30 fans are INSANE. They move an insane amount of air with such little noise this rig has been cooler than my previous machine that only had 2 3090s in it and I know it has to do with those fans

If you have any specific questions I may have left out don’t hesitate to ask

[-]

x8code@reddit

WTF you have 4x RTX 3090s in one system? That is awesome.

[-]

WyattTheSkid@reddit (OP)

I thought it was pretty dope too 😎

[-]

QuantumCatalyzt@reddit

I hope it serves you well. It looks beautiful

[-]

WyattTheSkid@reddit (OP)

Thanks! I just woke up and im gonna go put more fans in it right now :)

[-]

Glad-Programmer-5505@reddit

Damn nice

[-]

TheRenegadeKaladian@reddit

What is the motherboard you're using ?

[-]

WyattTheSkid@reddit (OP)

Asus ROG STRIX X570-E

[-]

libregrape@reddit

That 3090 ti looks like it's holding on for it's dear life. I would be so terrified to even breathe near it, fearing it will drop on the floor.

Great build though! And the spacing between everything looks generous enough that I wouldn't be worried about cooling. So, happy inferencing, OP!

[-]

WyattTheSkid@reddit (OP)

I promise it’s very secure. I took a vertical mounting bracket, cut a slot off it with a dremel, screwed it and ziptied it into the case, and then screwed the card in to it too

[-]

buildskate@reddit

Nice!

[-]

lucsaddler@reddit

Qual o setup, pode detalhar? CPU, Mobo, RAM, SSD, Fonte; Como conectou duas 3090, quanto de VRAM disponivel tem, sao 48GB, certo? Achei bem interessante, o que daria para rodar nesse setup, tendo em vista que o modelo ficaria dividido em duas gpu's?

[-]

QuantumCatalyzt@reddit