If it works - don’t touch it: COMPETITION
Posted by awfulalexey@reddit | LocalLLaMA | View on Reddit | 109 comments

Come on, share your "weird" home inference system builds. Let's have a little friendly competition. I think I am the absolute leader.
I took the grill from my wife’s oven, and I also found an egg carton.
I will design a new case, but not now.
If it works - don’t touch it.
4x3090, 128GB DDR4, 18/36 Cores
MagentL@reddit
2x 5060TI 16G + 4x MI50 32G
500w idle, 1.5-2kw under load
jacobpederson@reddit
Ok you win :D Mine is a masterpiece compared to this :D
SexyAlienHotTubWater@reddit
Jesus fucking christ
jacobpederson@reddit
Got to 80GB with mostly closet stuff and a $50 "case" :D
milpster@reddit
My proud little shitbox right here. Ryzen 5900hx, 64gb ram, 3080 max q 8gb and a 4x external pcie riser with a really worn down rx570 8gb. The laptop sitting on top of a fan box with a dust filter.
SexyAlienHotTubWater@reddit
Despite the size this is one of the more horrifying setups here well done
Fabulous_Fact_606@reddit
Just 2x3090 to run Qwen3.5-27B-AWQ-BF16-INT4
The single blower 3090 is a freakin jet engine and blows hot air @ 100% GPU. Had to move it to the garage.
Guess where the heat exaust is pointing towards? Hot water heat pump. It takes in the hot air to heat water and blows out cold air to cool the GPUs.
arcanemachined@reddit
Have you experimented with lower power limits? You might lose e.g. 10% performance but use 50% power (and generate 50% less heat).
lmdz6oz@reddit
That fan is chilling
k_means_clusterfuck@reddit
+loose fan
+tucked usb wifi
-water cooling
is that a single stick of ram?
Damn
No_Block8640@reddit
Have you tried quantized versions? I am wondering if the bf16 if just much more reliable than q4,q6 for example? Does bf16 make a quality difference?
Fabulous_Fact_606@reddit
That is a good question to investigate. BF16 looks good on paper and fits my 2x3090 with vlllm so I just went with it.
meganoob1337@reddit
it is "Q4"(int4) just some important layers are bf16 for better accuracy"
FluoroquinolonesKill@reddit
Indeed, mounting fans is a pretentious waste of time.
Leather_Flan5071@reddit
I don't have a pic but my laptop had a finicky and flaky Windows install that I attributed to, you know, windows. So I installed an old version, a new version, and linux, and it still had the same issue of randomly doing a BSOD or in linux, just shutitng down?
After a few days of troubleshooting I found that applying pressure on the middle of the SSD besides the screw made the laptop stable, so much so I had to put in a heatsink so when I put the backplate it gets pressed down.
It worked for a year or two
kuyermanza@reddit
8x 16GB Instinct MI25s clinging to life via PCIe x1-to-4-x1 splitters and a 12GB RTX3060, Ryzen 5 5500, 32GB DDR4, high-end custom cooling (central AC + cardboard duct).
ComplexType568@reddit
what sorta numbers do you get, and how did you get the 3060 working with the MI25s?
kuyermanza@reddit
Right now I use 5 MI25s for GPT-OSS 120B at ~30 TPS, 2 for Gemma 4 26B at ~30 TPS as well, and the last MI25 for embedding models. RTX3060 is purely for Z-Image Turbo.
Sbaff98@reddit
What software stack are you actually using?
In x1 how is it possible to get th 30tps in the 120b token
DonkeyBonked@reddit
I think most people overestimate what it takes to transfer with pipeline parallelism. I have a 3090 in a Razer X Core eGPU case connected via Thunderbolt 3 to my Dell Precision 7050 which has a 16GB RTX 5000 in it and honestly, it's quite good and not nearly as much slower than my 4x 3090 all on PCI-E 4.0 x16 as I thought it would be.
kuyermanza@reddit
Ubuntu 2404 and the rock 7.13 (nightly) as the backend and llamacpp for inference. Model loading phase is slow for sure, ~15 mins, but it shouldnt affect token generation since the bandwidth usage isn’t that high at this stage.
cunasmoker69420@reddit
hell yeah dude
toothpastespiders@reddit
The duct in particular is beautiful, like you made cardboard box fort for your build to play with. Though the garnish of tossing the power strip in a cardboard box too ties it all together perfectly. It really does have an oddly pleasing visual aesthetic.
ambient_temp_xeno@reddit
A thing of beauty is a joy forever.
This reminds me of District 9
9302462@reddit
Not janky server, but how about cooling, fiber and power, all held up by an Amazon shipping bag cut into strips.
https://imgur.com/a/eRjJPUW
This was when I had 4x3090’s in my office. It would trip the breakers because of other stuff in the cabinet as well as my desktop. It would also cause the room to be extra toasty with the door closed; heated floors are nice in the morning but sweating at your desk isn’t.
The power solution was a thick 25ft 12 gauge power cable running from an old hall closet where a furnace used to be through the air vent.
Cooling solution was a 4in dryer duct with a cardboard flap on it stuck in the air vent so it forces air down the tube. There was a Noctua fan with a 1inch gap between it and the tube which would help move the air across the GPUs; noctuas are volume not static pressure. The gap was so it could pull in cooler room air even when the AC wasn’t turned on and keep it flowing across things.
Fiber optic was between this cabinet and another I setup in a different room.
Ugly, yes. But it worked great and was out of site out of mind.
JeddyH@reddit
i5 2400, 8GB RAM.
Absolute beast.
maz_net_au@reddit
'tis a lovely pizza oven you've got there.
SexyAlienHotTubWater@reddit
This looks like the sarcophagus at Chernobyl
MoffKalast@reddit
3.6Ghz, not great, not terrible.
onewheeldoin200@reddit
*grabs the prayer beads and says three Hail Marys*
poop_magoo@reddit
Is that a stemless wine glass with mountain dew in the background?
Kahvana@reddit
The shed.
sine120@reddit
"Mom, can we have LackRack?"
"No, we have LackRack at home"
-LackRack at home:
SSOMGDSJD@reddit
Cardboard hangar shroud is underrated
swingbear@reddit
The cardboard is sending me 😂
arcanemachined@reddit
I literally laughed out loud when I saw that.
InternetExplorer9999@reddit
Looks great. I bet it can even run GLM 5.1 at fp16
awfulalexey@reddit (OP)
oh my god
llama-impersonator@reddit
it's beautiful
LightMaleficent5844@reddit
Remember if you're doing extended wordloads, half the vram uses the backplate as a heatsink. Can get well over 100c without good airflow. TJmax around 110c (?)
AdventurousLion9548@reddit
MoffKalast@reddit
Oi, you forgot to plug in your speakers into the stereo port.
Thrumpwart@reddit
Tell me more. Temps? Power draw? Inference speed?
Ok-Mess-3317@reddit
casually flexes his setup
Dundell@reddit
x6 RTX 3060's and added the P40 via zip ties due to missing a longer cable for it... It's been sitting like this for about 2 months now just fine. We cleaned up this office a few days ago and made sure she didn't touch my corner server.
toothpastespiders@reddit
Hell yeah, that's beautiful. I love jank builds.
poop_magoo@reddit
Is that a stemless wine glass with mountain dew in the background?
a_beautiful_rhind@reddit
I got all this shit for wood (still works btw) and this guy is using paper!?
Kahvana@reddit
Pics or it didn't happen!
a_beautiful_rhind@reddit
It definitely happened: https://i.ibb.co/B2pMJ9tc/Image100001.jpg
Kahvana@reddit
Very nice!
Long_War8748@reddit
Is there no cheap piece of metal in the house that can replace that very flammable egg carton 😅?
ciprianveg@reddit
16x3090 in 3 systems connected by 100gbit network. 2x6 3090 in the right (p620 based) and a p620 with 4 3090 in the left
ciprianveg@reddit
Rich_Artist_8327@reddit
what brand mcio risers you have?
ciprianveg@reddit
adt
Thrumpwart@reddit
Damn! This is beautiful. Those P620's are a steal and now I'm debating whether I need a couple.
megadonkeyx@reddit
16x 3090s on drawers that look like they are about to collapse under the weight. sir, i salute your priorities.
unculturedperl@reddit
N100 + 16gb + 1tb, aww yeah. Gemma4 E2B at 7.5tk/s, super fast. I should rename it the DGX Spork.
Shouldn't your top two cards be oriented vertically to allow for more airflow across the backsides? Can probably source another egg crate or two to make that work..
Kahvana@reddit
I wonder, how well does your N100 run small moe models like marco mini?
https://huggingface.co/AIDC-AI/Marco-Mini-Instruct
unculturedperl@reddit
It's mostly for testing purposes, so I don't expect it to do much useful work. Whatever it does load, it will run, slowly.
bombastic6339locks@reddit
I dont understand what you guys need these for. Kirkengeniuenly asking. How big is the difference between this and just whatever llm is near easiest to run
davl3232@reddit
robertpro01@reddit
Which mobo?
This is mine:
3x 3090, 1 4060 (to be replaced by another 4090) 32gb ddr4 Xeon 2680 v4 (40 lines) Mobo x99 asus
awfulalexey@reddit (OP)
ASUS X99 EWS
robertpro01@reddit
Nice, we have the same then
awfulalexey@reddit (OP)
ur setup too accurate
bad_detectiv3@reddit
Tbh I want to do this but is your quality better then just using open router or kilo code gateway to have access to say glm 5 and minimax for free?
-dysangel-@reddit
https://i.redd.it/i8ci6o1r06vg1.gif
truthputer@reddit
Your "wife's oven"?
Please tell me you're married to a professional chef who refuses to let you in the kitchen, or that seems like a remarkably sexist statement.
Any man who can't cook and can't find their way around a kitchen has no survival skills.
-dysangel-@reddit
I bet I could find my way around the kitchen, if I knew where it was
thejacer@reddit
Womp womp
awfulalexey@reddit (OP)
oh shi\~
DonkeyBonked@reddit
Pretty similar to the OP 4x 3090, 128GB RAM. EPYC 7502 HUANANZHI H12D-8D Motherboard.
SSOMGDSJD@reddit
RGB fans set to red for decode tok/s boost of course. 10/10 ambiance
awfulalexey@reddit (OP)
Hmmm nice idea
Imaginary-Anywhere23@reddit
Have you guys think of replacing the boiler with graphic cards?
Kahvana@reddit
If you replace the egg holder and cardboard with a metal can, you might be able to heat food in it!
AdamDhahabi@reddit
The left GPU is a P5000 which I replaced with an RTX 3090.
Second power supply in the right.
Ok-Welcome-3750@reddit
You made my day, thanks 😂
MoneyPowerNexis@reddit
https://imgur.com/a/L2e3sDj
Reggitor360@reddit
Currently at work so no pic but here is the hardware at least.
9800X3D, 96GB DDR5 6000 CL28, 7900XTX waterblocked and 900W PL, Ai Pro R9700 AI TOP.
And yes, I game on it during training, thats why the XTX is still there xD
roselan@reddit
I love these setups.
Out of curiosity, did you guys install CEE connectors or do you yolo it?
MiF36rus@reddit
-dysangel-@reddit
my janky ass setup
awfulalexey@reddit (OP)
booooring 🥱
-dysangel-@reddit
:p
-dysangel-@reddit
how fast does it grill the bacon?
Rich_Artist_8327@reddit
2x 5099 1x 7900 xtx
Substantial-Ebb-584@reddit
It looks neat. What case is this?
Rich_Artist_8327@reddit
It takes 3 PSUs and 4 3slot GPUs
Substantial-Ebb-584@reddit
Yeah it's nice for multiple GPUs. But the name/symbol/link please?
_supert_@reddit
This is like /r/llamagonewild
FPham@reddit
Is there a carton of eggs propping your GPU?
awfulalexey@reddit (OP)
Yes. This is a special technical solution to connect power to the GPU in this situation, and to avoid interfering with another one nearby, a cascade is needed.
denoflore_ai_guy@reddit
This thread brings me joy.
awfulalexey@reddit (OP)
you are well cum!
denoflore_ai_guy@reddit
Custom rack. Roll out motherboard tray. It’s still in build process but it works it’s fast and I learned a lot building it so I’m good. lol. Fuck I need to clean my basement. 😑
HeavyMike@reddit
I think the eggs will make a pretty good heatsink but you will need to replace them before they go rotten
Ok-Palpitation-905@reddit
Behold the T620
awfulalexey@reddit (OP)
oh yes
Karnemelk@reddit
this is why i joined this sub
Snoo_28140@reddit
LlamaPorn 🤤
llama-impersonator@reddit
you run out of zip ties mid-build or somethin'?
FissionFusion@reddit
Dude's got the ROG Strix High Performance Cookie Rack Platinum
Lesser-than@reddit
egg carton, coming in clutch.
CATLLM@reddit
Work of art.
Makers7886@reddit
Brings me back to the good ole mining days. I had a wire metal shelf with 12x3090s hanging by zipties and box fans blowing on it. One day I got fancy and put them in open air mining chassis where they still live. I never could take a picture of the wire shelf situation. Like parking an exotic car in a muddy field.
koushd@reddit
beautiful