This is a sequel to my previous thread from 2024.

I originally planned to pick up another pair of MI100s and an Infinity Fabric Bridge, and I picked up a lot of hardware upgrades over the course of 2025 in preparation for this. Notably, faster, double capacity memory (last February, well before the current price jump), another motherboard, higher capacity PSU, etc. But then I saw benchmarks for the R9700, particularly in the llama.cpp ROCm thread, and saw the much better prompt processing performance for a small token generation loss. The MI100 also went up in price to about $1000, so factoring in the cost of a bridge, it'd come to about the same price. So I sold the MI100s, picked up 4 R9700s and called it a day.

Here's the specs and BOM. Note that the CPU and SSD were taken from the previous build, and the internal fans came bundled with the PSU as part of a deal:

128GB VRAM, 128GB RAM for offloading, all for less than the price of a RTX 6000 Blackwell.

Some benchmarks:

A few final observations:

• glm4 moe and minimax-m2 are actually GLM-4.6V and MiniMax-M2.1, respectively.
• There's an open issue for Qwen3-Next at the moment; recent optimizations caused some pretty hefty prompt processing regressions. The numbers here are pre #18683, in case the exact issue gets resolved.
• A word on the Q8 quant of MiniMax-M2.1; --fit on isn't supported on llama-bench, so I can't give an apples to apples comparison to simply reducing the number of gpu layers, but it's also extremely unreliable for me in llama-server, giving me HIP error 906 on the first generation. Out of a dozen or so attempts, I've gotten it to work once, with a TG around 8.5 t/s, but take that with a grain of salt. Otherwise, maybe the quality jump is worth letting it run overnight? You be the judge. It also takes 2 hours to load, but that could be because I'm loading it off external storage.
• The internal fan mount on the case only has screws on one side; in the intended configuration, the holes for power cables are on the opposite side of where the GPU power sockets are, meaning the power cables will block airflow from the fans. How they didn't see this, I have no idea. Thankfully, it stays in place from a friction fit if you flip it 180 like I did. Really, I probably could have gone without it, it was mostly a consideration for when I was still going with MI100s, but the fans were free anyway.
• I really, really wanted to go AM5 for this, but there just isn't a board out there with 4 full sized PCIe slots spaced for 2 slot GPUs. At best you can fit 3 and then cover up one of them. But if you need a bazillion m.2 slots you're golden /s. You might then ask why I didn't go for Threadripper/Epyc, and that's because I was worried about power consumption and heat. I didn't want to mess with risers and open rigs, so I found the one AM4 board that could do this, even if it comes at the cost of RAM speeds/channels and slower PCIe speeds.
• The MI100s and R9700s didn't play nice for the brief period of time I had 2 of both. I didn't bother troubleshooting, just shrugged and sold them off, so it may have been a simple fix but FYI.
• Going with a 1 TB SSD in my original build was a mistake, even 2 would have made a world of difference. Between LLMs, image generation, TTS, ect. I'm having trouble actually taking advantage of the extra VRAM with less quantized models due to storage constraints, which is why my benchmarks still have a lot of 4-bit quants despite being able to easily do 8-bit ones.