At Home LLM Build Recs?
Posted by MintiaBreeze1@reddit | LocalLLaMA | View on Reddit | 8 comments
Hey everyone,
New here, but excited to learn more and start running my own LLM locally.
Been chatting with AI about different recommendations on different build specs to run my own LLM.
Looking for some pros to give me the thumbs up or guide me in the right direction.
Build specs:
The system must support RAG, real-time web search, and user-friendly interfaces like Open WebUI or LibreChat, all running locally on your own hardware for long-term cost efficiency and full control. I was recommended to run Qwen2.5-72B and other models similar for my use case.
AI Recommended Build Specs:
GPU - NVIDIA RTX A6000 48GB (AI says - Only affordable 48GB GPU that runs
Qwen2.5-72B fully in VRAM)
CPU - AMD Ryzen 9 7950X
RAM - 128GB DDR5
Storage - 2TB Samsung 990 Pro NVMe
PSU - Corsair AX1000 Titanium
Motherboard - ASUS ProArt X670E
I have a server rack that I would put this all in (hopefully).
If you have experience with building and running these, please let me know your thoughts! Any feedback is welcomed. I am at ground zero. Have watched a few videos, read articles, and stumbled upon this sub-reddit.
Thanks
PraxisOG@reddit
Even if you’re set on running locally, I’d recommend checking out some models on openrouter or a similar service just to make sure you’re happy with the performance you’re going to get before spending thousands of dollars on hardware.
Rich_Repeat_22@reddit
NVIDIA RTX A6000 48GB buahah (the laughing is to the LLM who gave you the answer).
Get 2 R9700 32GB ($1300 each). 64GB VRAM could easily fit in any mainstream board too, and for 2/3 the price of the A6000 which is $4000-$6000 used.
Get a motherboard supporting 8x8 (8 PCIe 5x lanes per GPU when 2 are used) from AMD 670E/870E/670 lineup.
Not all do, there is an application to check which are the ones.
CPU & RAM take your pick. 9950X is good but if you plan more than inference (eg gaming) etc get a 9950X3D, or a 9800X3D. 128GB is good enough preferably 2 sticks of any speed over 6000. (don't try to go down the 1-1-1 route, just get fast RAM).
Makes sure you buy ATX3.1 PSU with 2x16pin connectors, DO NOT try to use ATX3.0 with 8pin to 16pin converters.
ga239577@reddit
My tip is don't waste your cash on it. Unless you have tons of cash to burn or something.
AppearanceHeavy6724@reddit
Dammit do not listen to LLMs. 2x3090 will run Qwen 2.5 72b easy peasy.
Qwen 2.5 is ancient (and is not that good by standards of late 2025), I'd rather run glm or OSS 120B in RAM the days.
Optimalutopic@reddit
You may be interested in this: https://github.com/SPThole/CoexistAI
previse_je_sranje@reddit
That's a nice rack
Marksta@reddit
If you already have a server rack, maybe consider the AMD MI50 32GB route. Otherwise you're looking at multiple 3090s, 5090s, or the rtx 6000.
Dont listen to AI, use the search tool and look at other people's builds and there's a sticked hardware thread too.
cbale1@reddit
Is 128GB RAM needed if the model can be fully run on VRAM?