Ram-air setup and window vent for 1100w capable AI box
Posted by mr_zerolith@reddit | LocalLLaMA | View on Reddit | 91 comments
So i have a very powerful setup here and i got tired of the office being a sauna.
Here is my solution and the cooling achieved is as effective as being open case.. about 90% of the heat makes it out the window.
Shared hoping that it serves as some inspiration to others. Cheers!
CalligrapherFar7833@reddit
With that large vent on the side panel and that long of a exhaust you are losing shit ton before venting
mr_zerolith@reddit (OP)
I have opened the case only so that reddit can see the inside. It is normally closed.
CalligrapherFar7833@reddit
Vent on the side is on the side panel
mr_zerolith@reddit (OP)
That's normally covered with a book when the case is on
CalligrapherFar7833@reddit
Ok then only thing thats left is the long exhaust run
Kahvana@reddit
Thanks for sharing, looks crazy!
What recommendations would you make to someone trying to run a setup like yours?
What works well, what would you do different?
mr_zerolith@reddit (OP)
Everything worked well, the only changes i'd make:
- Locate the box above the vent so that rain cannot come in if i fail to close the window.
- 140mm exhaust fan instead of 120mm and 5 inch drier hose instead of 4 inch so that the fan RPM, and thus noise, could go down a bit.
- Less bend on the drier hose
At some point i'd like to sell the 5090 and get another RTX PRO 6000, because moar power is always better
deepspace86@reddit
FYI having the intake be that close to an open window basically means you're going to be pulling in air containing a fair amount of moisture.
mr_zerolith@reddit (OP)
It's constantly blowing air outwards even at idle and the office is really dry inside ( \~22% humidity, we're in the high desert in Utah, USA )
It rained the other day and i didn't notice it was more humid in the office ( i have meters for that )
It would be strange for humidity to enter when we're constantly blowing out dry air.
Could also just shut the window on those rare rainy days and deal with the sauna factor again.
barnett9@reddit
He means coming in the window not the duct from pressure differential
mr_zerolith@reddit (OP)
I get what you mean, i can just close the window if it's substantially windy
Acceptable-Yard7076@reddit
Not really something to worry about in the southwest
Ambitious_Worth7667@reddit
exactly......case and point: Swamp coolers.
sourceholder@reddit
Air liquid cooling.
ambient_temp_xeno@reddit
Increase your mmproj max tokens.
CokeAndChill@reddit
Recommendations:
Case above the intake level, avoid water or whatever coming in.
Also pull air from outside in the summer, so you don’t exhaust AC air. You can add a cheap HEPA filter to the intake to keep it clean.
mr_zerolith@reddit (OP)
I agree, i'll add that in!
Xamanthas@reddit
https://www.arctic.de/en/S12038-4K/S12038
Consider this fan, lol.
mr_zerolith@reddit (OP)
Damn!
You know 3000 RPM fans are pretty loud already!
Xamanthas@reddit
There’s an 8K version xd
tavirabon@reddit
Wouldn't hurt to put another fan at the other end of the duct, keep the pressure gradient more even and reduce inner turbulence to increase airflow efficiency. If you can get an IP67 fan with the same CFM as your big heavy german fan, anyway.
bnm777@reddit
What do you use it for?
mr_zerolith@reddit (OP)
I run a web development shop and have 8 developers who use it.
Our clients demand privacy because all of them have unique software as their moat so we can't use most AI systems. Also, we found public AI systems to be too unreliable.
We don't do a lot of agentic work, mostly use CLine or just basic code generation. Everyone is on different time zones, so having 2 parallel requests available is just enough capacity.
ambient_temp_xeno@reddit
Cardboard and zip ties truly are the cyberpunk materials that everyone converges on.
contact@reddit
I see NEITHER of those things in this post.
NuScorpii@reddit
Bottom middle annotation
contact@reddit
Doh! My apologies.
I stand corrected.
ambient_temp_xeno@reddit
NuScorpii@reddit
Bottom middle annotation
takoulseum@reddit
Is the window open?? If so, no need for this big snake then?
Ambitious_Worth7667@reddit
I usually don't spit coffee out....
but the thought of all that and the window being closed did it this morning.
Congratulations
mr_zerolith@reddit (OP)
The window is partially open.
If we don't focus the air out the window, the office becomes a sauna, and even worse, the box is intaking hot air now from the surrounding environment.
Ambitious_Worth7667@reddit
that looks really close to bitcoin mining back in the day....
laterbreh@reddit
Inference stack? Llamacpp?
TRKlausss@reddit
Plug a water-water heat pump, get hot water for free*!
dbenc@reddit
use another hose to bring in fresh air (with a filter) or you're just pumping your a/c out the window
SkyFeistyLlama8@reddit
Cold air intake from the car modding world.
If you're running inference on a laptop like a llama-idiot, just like me, you can use a large case fan powered over USB. Make an airflow duct from cardboard to point specifically at the CPU or GPU block on your laptop. Don't point it at the laptop fan because those tiny fans don't like working in negative pressure.
mr_zerolith@reddit (OP)
Ah it's more like a muffler since that pipe is the exhaust.
Because \~90% of the hot air gets mostly shot out of the window, the intake is cool air from the air conditioned apt!
( i have thought about making an intake part that's next to the AC vent, lol )
SkyFeistyLlama8@reddit
If you really want to geek out, try looking at the airflow rate out the exhaust duct and design something that's close to a straight pipe. Don't need torque on a computer case LOL.
unculturedperl@reddit
What's the sound level of this?
The tight bend in the exhaust pipe is also limiting your effectiveness.
yaosio@reddit
Bends reduce airflow. When the air hits that first bend some of it will bounce backwards and run into the air coming up the tube. If you're able remove the bend or reduce the angle of the bend.
mr_zerolith@reddit (OP)
Yeah i know, it performs well though thermally despite that. :)
ShelZuuz@reddit
Your entire box was $13k??
Or just the GPU's?
Actually, even then.
mr_zerolith@reddit (OP)
GPUS = $12k
Other parts were bought cheap or open box on ebay.
Weak CPU and ram because we basically won't use it
caetydid@reddit
Congrats. This is really cheap!
AIDaemon@reddit
Its crazy and I like it!
Comms@reddit
Inline duct fans are super quiet and move air like whoa.
FearFactory2904@reddit
Doesnt ram air mean having a vent on the front of a vehicle so that when you drive forward the movement creates enough air pressure to "ram" the air down the intake piping? So for this to be ram air you need to drive your house fast enough with that window forward facing so that it creates the needed air pressure.
mr_zerolith@reddit (OP)
Ok, I'll work on it!
crantob@reddit
It's a simple good idea and a good test to identify the 'conventionally minded' here.
flockonus@reddit
It's cool, but watch out for DUST in the intake and accumulation in the card.
Dust will accumulate at whatever multiple is your intake volume.
mr_zerolith@reddit (OP)
Yep, i know, this machine and me live in the high desert and dust is a constant menace. I blast everything out every 2-3 months.
crantob@reddit
Did you ever read-up on compressed air and electronics and whether that's a good idea or not?
Dos-Commas@reddit
But why, you can run it at 124 tokens per second at FP8 for 30 cents per million tokens on the cloud.
https://openrouter.ai/stepfun/step-3.5-flash
mr_zerolith@reddit (OP)
thanks but no thanks
Dos-Commas@reddit
To research and their own, it'll literally take you 29 years to breakeven.
mr_zerolith@reddit (OP)
No thanks
Sliouges@reddit
This is why I come here. Awesome. You need a few dozen paper pulp egg-boxes for sound proofing, they absorb sound pretty well.
bigginz87@reddit
How is the coil whine on the 6000 with that overclock?
mr_zerolith@reddit (OP)
You know, stock i can hear it with the case open.
It's a little louder if you just add 200mhz.
It's barely louder at 1500mhz.
This card also contains memory capable of +3ghz versus stock. I think they may have downclocked the memory to lower the coil whine.
bigginz87@reddit
Good to know. The coil whine has been really annoying me lately, but I guess it's a sacrifice worth making.
mr_zerolith@reddit (OP)
Vram overclocking is so worth it.
I'm told coil whine is harmless by some knowledgeable OCers
zilled@reddit
where is the air intake?
if this is from the outside, careful about having the input air colder than the inner dewpoint -> condensation -> everything water-fried.
mr_zerolith@reddit (OP)
Please refer to the arrows and text in the image
zilled@reddit
sorry, missed that.
honestly, I would suggest full watercooling, with a MO-RA outside.
No noise, no heat.
(just be careful about low winter temps that could create condensation).
lowrizzle@reddit
Server cooling is a lot more complex than 'just push a lot of air through it really fast' but so long as you're happy with it that's what really counts, i guess.
mr_zerolith@reddit (OP)
Yep, you need to ideally direct air flow, etc, which i've done here, and we can keep below 75C easily.
It's roughly equivalent to having an open case in terms of cooling due to the high velocity of air movement.
dinerburgeryum@reddit
Peak r/LocalLLaMA right here folks.
PhotographerUSA@reddit
Just wait, until the creatures from the outside come in there lol
mr_zerolith@reddit (OP)
nah, i have a screen and i'm on the third floor, maybe at worst a bird will perch there to enjoy the heat
abnormal_human@reddit
This is ... a lot for 1100W.
mr_zerolith@reddit (OP)
You would be shocked at how much heat these cards make, even stock..
braydon125@reddit
Bro you spent 13k $ on that?
mr_zerolith@reddit (OP)
Yup it's approximately as cheap as you could pull off to get this level of intelligence
StandardLovers@reddit
This is almost /stonerengineering
mr_zerolith@reddit (OP)
But it's simultaneously blue collar pilled
StandardLovers@reddit
Dont toast you cards with overclocking in a bad case design.. whatever; you'll figure it out. Nice problem solving skills, I like the arrows.
starkruzr@reddit
only partially related to the topic but how do you manage the asymmetry of RAM and compute between those two cards?
mr_zerolith@reddit (OP)
Oh, that's no problem.
I use LMStudio and the 'priority split' option.
So we end up with the RTX PRO 6000 mostly filled and the 5090 ( which is slower ) a bit more empty by %
It figures out the rest.
it's more ideal to run llama.cpp with CUDA compiled in but i can't figure out how to do that. Then you can dial in a more precise split and favor the faster GPU more for a boost in speed.
But the way LMStudio does it is good enough!
One-Replacement-37@reddit
Meanwhile I am rawdogging a 10kW AI setup in the basement in proper server racks, never had heat issues...?
mr_zerolith@reddit (OP)
I'm in an apartment in the top floor and my desk is right next to that thing.
housepoor problems
One-Replacement-37@reddit
top floor is rough as hell in the summers, sorry op!
mr_zerolith@reddit (OP)
Hey at least there's a solution!
Professional-Safe894@reddit
Cooles setup. Ich selbst experimentiere gerade mit einem Dgx spark. Was machst du damit ? Also es muss sich doch finanziell für dich lohnen ? Oder wirklich nur ein teures Hobby ?
ShelZuuz@reddit
"I see your Noctua and raise you 'Big heavy German made fan' ".
mr_zerolith@reddit (OP)
More specifically:
https://www.arctic.de/us/P12-Pro/ACFAN00305A
It is pretty heavy compared to Noctuas i've used.
I bought it primarily because i needed 3000RPM max, 2000RPM was not cutting it when the case was closed.
nmrk@reddit
Ventilation fans and ducts are cheap lately, due to the indoor growing industry.
Local_Phenomenon@reddit
My Man! Incredible setup.
koushd@reddit
did you put a hole in the window's glass or what?
bornlasttuesday@reddit
I think he used his ai box to make this picture
mr_zerolith@reddit (OP)
Nah, i just open the window a hair and leave the blinds open.
The high velocity of air coming out means that a majority of the air makes it out the window.