Local LLM Server with Radeon AI Pro R9700

Posted by Niehaus_1301@reddit | buildapc | View on Reddit | 3 comments

Hey PC nerds,

I need your confirmation if you accept me as being a good PC nerd too.

I want to build a local LLM server running Hermes 4.3 36B model.
This is what I thought is good for my budget:

CPU: AMD Ryzen 9 9900X
GPU: Gigabyte Radeon AI PRO R9700 AI TOP 32GB GDDR6 PCIe 5.0
RAM: G.Skill Trident Z5 Neo 64GB (32GBx2) DDR5 6000Mhz
Cooling: Thermalright Peerless Assassin 120 SE CPU Air Cooler
Storage: WESTERN DIGITAL SN580 2TB (I already own this and thought I could use it to get started)
PSU: CORSAIR RM1000x Low Noise Fully Modular ATX
Case: Phanteks XT Pro Tempered Glass Windows - Black

Total cost is just under €3000

Ideally I want to use vLLM to run multiple Hermes Agents in parallel, perhaps with multiple chat sessions happening simeoultaneously. Don't really need it for Coding, I have a Claude Max subscription for that.

What do you think about this? Anything you'd swap? I used mostly AI (Opus) to define this setup.

Thanks for your advice!