Dell T5820 w/ 2x Dell RTX 3090 for less than $2k - eBay sourced

[-]

Kenavru@reddit

also got 2 of those dell 3090, check temps on reverse GDDRs, those get hot AF, also GPU FAN doesnt care if they are frying if GPU isnt much loaded (in LLM usage when got memory 100% and GPU 10%, fans didnt even start !)

[-]

_Boffin_@reddit (OP)

in LLM usage when got memory 100% and GPU 10%, fans didnt even start !

interesting... did you do any configuration to get the fans to start running or just point another fan at it and say, go brrr..?

[-]

Kenavru@reddit

Nah, fan speed worked only on arch for me,

[-]

Palladium-107@reddit

Can you tell me what model / brand these 3090's are? These look perfect sizewise. I have T7920 that I want to upgrade with a set of 3090's

[-]

_Boffin_@reddit (OP)

They're the Dell OEM RTX 3090 cards. It says they're 2 slots, but they're not. 2.5 slots.

[-]

Palladium-107@reddit

Yeah these are perfect because they are 2 slot, which is hard to find and better than the 2-slot turbo variants. That is a good deal.

[-]

mj3815@reddit

Curious why you say they’re better than turbos? I have a turbo and I’m trying to find a second slim (2 slot) card to add.

[-]

_Boffin_@reddit (OP)

From things i've came across, which ain't much and i forget where i found it or what it said... heating issues, i think?

[-]

mj3815@reddit

Mine is installed directly on top of my 3080. Power limited to 285w. Under sustained load I see up to 80C on the chip and 96C on the RAM with the default fan curves. I typically override and set fan to 100% and all those temps drop 10C. It’s loud as hell though.

[-]

Palladium-107@reddit

The 3090 cards have 24 GB of VRAM of which 12GB is the backside of the PCB. While the Turbo's do their job for what they are, they are pretty loud and they don't provide not so good cooling to the backside of the pcb. So idealy you would want to have a 3090 with a real backplate keeping the vram's better cooled. In case of inferencing this is of a lesser issue. I would try to undervolt the Turbo variants instead of only lowering the tdp, but that also depends on the case you are using how god the airflow is.

[-]

_Boffin_@reddit (OP)

2.5 slots.

[-]

Palladium-107@reddit

Thx

[-]

salec65@reddit

Are these blower style coolers that force hot air out the back or do the fans blast hot air through the chassis?

[-]

_Boffin_@reddit (OP)

I believe it's not a blower style. Here's a teardown: https://www.youtube.com/watch?v=_8k5PCmFm7c

[-]

PermanentLiminality@reddit

Can you measure the idle power draw?

[-]

rorowhat@reddit

What are the system specs?

[-]

salec65@reddit

There are 3090's that only take 2 PCIe power connections and are proper 2 slots? My EVGA monster was 2.5 slots, took 3 power connections and dumped all of the hot air back into the chassis. No way I'd have 2 of them next to each other.

Do these have blower style coolers on them?

[-]

_Boffin_@reddit (OP)

After getting the wrong cables a few times, it finally is up and running.

Limited the wattage to 285 per gpu.
This has a 950w PSU.
Not running anything else at the moment
No heat issues yet.

Have the first gpu in slot1, which is a x8 and have the second one in slot 4, which is a x16.

With no tuning, this is what i'm getting for a 70b q3 on first message.

response_token/s: 14.75    
prompt_token/s: 282.61    
total_duration: 37825099415    
load_duration: 35064409    
prompt_eval_count: 26    
prompt_eval_duration: 92000000    
eval_count: 556    
eval_duration: 37696000000    
approximate_total: "0h0m37s"

[-]

largemachinery@reddit

Hey there! I'm nearly in this exact use care. What cables finally ended up working for you? I'm having a lot of trouble finding definitive information on the internet, and of course every seller claims their cable to work...

[-]

_Boffin_@reddit (OP)

10pin to 8+8pin Power Cable for DELL Precision 5820 and GPU 63cm from hk-estreet on eBay.

All my 8pin coming from my PSU were already taken up so i had to use the 10pin, which was the only remaining connector.

[-]

kryptkpr@reddit

Looks so clean. How's temps? I assume those are some massive intakes in behind the cards keeping them fed..

[-]

_Boffin_@reddit (OP)

I haven't seen temps above 74c yet for both, but i only played around for a few minutes as i watched the temps. through the BIOs, i also set the fan (to the right of the gpus) to always be going at around 5% at min.

Will add an beat to drain Nvidia's SMI info to Elastic soon to get a better idea of what's happening when doing work on it.

[-]

kryptkpr@reddit

Not sure about beats but there's a great official Prometheus exporter: https://github.com/NVIDIA/dcgm-exporter

Would be curious to see how tensor parallel batch with tabbyAPI or vLLM looks, it should push both cards to their power limit together and really stress cooling.

[-]

_Boffin_@reddit (OP)

I'll have to look into both things you mentioned. Do you think that due to the differences in the x8 / x16, the'll be any issues for the tensor parallelism?

[-]

kryptkpr@reddit

x8 is fine for inference, if you're planning on fine-tuning an nvlink bridge might help.

[-]

_Boffin_@reddit (OP)

this rig will be just for inference. if i wanted to finetune, i'll be going with an online provider for gpus.

[-]

atineiatte@reddit

That's pretty slick. I had to drill the rivets out of my HP Z440 case bottom to bend it out of the way of the second 3090

[-]

_Boffin_@reddit (OP)

thanks! that's some impressive surgery you did on the machine. How's it fairing?