Local LLM Server with Radeon AI Pro R9700
Posted by Niehaus_1301@reddit | buildapc | View on Reddit | 3 comments
Hey PC nerds,
I need your confirmation if you accept me as being a good PC nerd too.
I want to build a local LLM server running Hermes 4.3 36B model.
This is what I thought is good for my budget:
CPU: AMD Ryzen 9 9900X
GPU: Gigabyte Radeon AI PRO R9700 AI TOP 32GB GDDR6 PCIe 5.0
RAM: G.Skill Trident Z5 Neo 64GB (32GBx2) DDR5 6000Mhz
Cooling: Thermalright Peerless Assassin 120 SE CPU Air Cooler
Storage: WESTERN DIGITAL SN580 2TB (I already own this and thought I could use it to get started)
PSU: CORSAIR RM1000x Low Noise Fully Modular ATX
Case: Phanteks XT Pro Tempered Glass Windows - Black
Total cost is just under €3000
Ideally I want to use vLLM to run multiple Hermes Agents in parallel, perhaps with multiple chat sessions happening simeoultaneously. Don't really need it for Coding, I have a Claude Max subscription for that.
What do you think about this? Anything you'd swap? I used mostly AI (Opus) to define this setup.
Thanks for your advice!
seancourage23@reddit
Any updates? I want to build a similar local server.
Reggitor360@reddit
Throw out the AsRock board.
They suck for multi GPU.
Buy the X870E Pro Art Creator.
GPU, buy two R9700 and get a cheap Ryzen 7 7700(X)/9700X, 9900X isnt scaling good enough to buy it.
Niehaus_1301@reddit (OP)
Thx, appreciate! I switched the motherboard