Intel Arc Pro B60? (In Quad... 6x... 8x configuration)

Posted by Infinite100p@reddit | LocalLLaMA | View on Reddit | 34 comments

Has anyone tried running multiples of Intel Arc Pro B60 with 24GB VRAM with larger models like MiniMax, maybe quants of GLM? Would it be a good budget choice at \~$650 per GPU given that 3090 stock is very thin now and they go for much more with no warranty and most of the lifespan gone? It's hard to find eBay listings below $800 for 3090, and that will get you a (severely?) used GPU with no warranty. I only found [these benchmarks](https://www.storagereview.com/review/intel-arc-pro-b60-battlematrix-preview-192gb-of-vram-for-on-premise-ai) for a multi-B60 setup, but the numbers seem off, and [this discussion here blames the author](https://www.reddit.com/r/LocalLLaMA/comments/1pd3mdw/comment/ns37lg3/) aka the tests were probably not properly set up. Would love to check in if anyone has **new** data points/experience to report? I am considering a 6x B60 set up. Thanks

Reply to Post

34 Comments

[-]

Fit_West_8253@reddit

I’m pulling the trigger on one next week. There’s cpp images with the Intel driver already on board, so I’m hoping setup will be straight forward. I know it’s not going to be as fast as high end nvidia cards and potentially some trouble with making things work smoothly, but I can’t justify the nvidia prices these days.

[-]

Infinite100p@reddit (OP)

> There’s cpp images with the Intel driver already on board Could you please link me to the images you are talking about? Thanks!

[-]

damirca@reddit

It’s here https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/SYCL.md#docker You clone the repo, build the docker image with one command and it works. The issue is that llama.cpp is slower than llm-scaler (that is built on top of vllm), BUT latest version of llm-scaler is built on top of vllm 0.12 and only some models are supported, BUT llm-scaler is snappier and more mature than llama.cpp, BUT llm-scaler does not support smaller quants for most of the models (so lowest for qwen3 VL 8b is fp8 which takes 18gb vram with 16k context) . As you can see there are a lot of buts. The stack is not ready yet, I’m monitoring llm-scaler repo every day, they plan to release llm-scaler with vllm 0.13 “soon” so I hope I can run ministral or glm 4.6V 9b on it.

[-]

Timely-Bus8404@reddit

I've been eyeing these too but haven't pulled the trigger yet - the 24GB VRAM per card is tempting but Intel's ML drivers still feel pretty janky compared to CUDA That benchmark you linked was definitely sus, they were probably running some weird mixed precision setup that skewed everything For 6x cards you're looking at 144GB total which is solid, just make sure your board can actually handle that many PCIe slots without choking the bandwidth

[-]

Infinite100p@reddit (OP)

Thinking either: ROMED8-2T (with an Epyc 7003) 7 PCIe total but at 4th gen or ASUS TRX50 SAGE Pro WS WiFi CEB (with a Threadripper 9000) with bifurcation of one of the slots (it's got 5 total at 5th gen). Do you see any caveats with either of these paths? Thanks

[-]

TokenRingAI@reddit

The B60 is a PCIe 5.0x8 card, you should go with the newer generation Epyc or Threadripper for PCIe 5.0 support. Your best option is probably an Asus k14pa-u12 motherboard which has 8 PCIE 5.0x8 MCIO connectors. Will save you money vs buying a bunch of redrivers. Check out the bundles on ebay that include the 9654 QS

[-]

TokenRingAI@reddit

If they had any kind of decent performance, Intel would be flooding us with benchmarks instead of quietly selling these mostly to cost-concious OEMs, that tells you all you need to know.

[-]

reto-wyss@reddit

Why go 6 (8x) of those? You can go 4x R9700 32GB with close to same VRAM and it will fit nicely in a standard chassis and it will only be marginally more expensive (also saving on jank setup stuff)? I've been looking at them, but I can't spin it anyway where getting multiple of these is better than getting multiple R9700, which I will likely get to replace my aging 3090s.

[-]

NunzeCs@reddit

I have 4x R9700, the speed is not amazing. Maybe after some more software updates, rocm gets better but It still needs more optimisation

[-]

Infinite100p@reddit (OP)

What are your pp and tg numbers? What are the biggest models you are running?

[-]

NunzeCs@reddit

at the moment I have qwen 3 235B in Q3_K_XL running with llama.cpp and rocm 7.1.1. In real use, 20-25 tokens/s and 500-800 tokens/s.

[-]

Infinite100p@reddit (OP)

Have you tried quants of GLM4.7, MiniMax? 500-800 is pp? At what context size? Thanks

[-]

NunzeCs@reddit

Context from 0-50k, I had tried GLM 4.7 REAP Q4_K_M and Q3_K_M, Q3 fits completely in VRAM. Was also about 20tokens/s for pp. But I am using it in German and I think because of REAP the models German Skills were distroyed. In English it run all fine

[-]

Infinite100p@reddit (OP)

Oh, I realize this is not a 5090, let alone 6000 pro, but 20 t/s for prompt processing is pretty low, no? You didn't perchance mean token generation? If not, what is your token generation? Thanks

[-]

NunzeCs@reddit

20t/s for token generation sorry, my English is not really good. REAP, where like 40% of the size of the experts gets removed. So the model is smaller. I am not quite sure about Q3, it feels fine.

[-]

Infinite100p@reddit (OP)

Thanks What is your prompt processing speed?

[-]

Infinite100p@reddit (OP)

Not marginally. The cheapest R9700 32GB I am seeing are $1358 for preorder and $1400 in stock. More than double the cost of 24GB B60 @ $650. What is the upside of 50-60%% worse cost per GB of VRAM besides saving slots?

[-]

reto-wyss@reddit

Cheapest R9700 I can buy from stock right now is CHF 1150, cheapest B60 is CHF 673 (for the passive card, blower appears out-of-stock). If prices are significantly different for you things may change, you also have to account for all the risers, time to set that up, etc. Fewer cards are always better, gives you more potential to expand if you want to go jank later. Slots are valuable.

[-]

Infinite100p@reddit (OP)

The FP16 TFLOPs are much better on R9700. I wonder how much it actually translates into real life difference.

[-]

Infinite100p@reddit (OP)

Yeah, in your case the math is different. In my case B60 would be $650, around $662 with shipping. R9700 can be had for $1,300. 2x difference, but also 50% more VRAM. So, 25% worse per GB but possibly better driver support? Hmmm. The commenter below says they are not very fast though. Arc would probably not be fast either though, but I can't find decent benchmarks for LLM on multi-GPU setups with a bunch of B60's.

[-]

reto-wyss@reddit

It's really hard to find good benchmarks. In some cases I see up to 70% performance of 5090 claims, others say it's slow. I don't know how well the Intel-Stack works, a months or so ago I tested Z-Image-Turbo on an A770 and that worked, but I haven't poked around much lately. From what I've seen, ROCm works great with vllm and that's likely what you want to run with 2 or 4 or 8 cards.

[-]

Fit-Produce420@reddit

NVidia is worth it just for the software stack.

[-]

m31317015@reddit

Won't downvote you but still this is a crazy wildcard statement.

[-]

LegitimateCopy7@reddit

still depends on the price. it's starting to get ridiculous.

[-]

Mundane-Light6394@reddit

check the memory transfer speed. If you batch (serve multiple clients) memory transfer is less important. If you solo it locally memory transfer is more important. These pro cards concentrate more vram per slot but their performance for unbatched workloads is underwhelming because of lower memory transfer speeds.

[-]

icanseeyourpantsuu@reddit

In terms of support, is this better than amd offerings?

[-]

ImportancePitiful795@reddit

There is a good review here [Intel Arc Pro B60 Workstation Graphics Card Review with technical Analysis and Teardown - Battle of the small Workhorses under 1200 Euros | Page 15 | igor´sLAB](https://www.igorslab.de/en/intel-arc-pro-b60-workstation-test-with-technical-analysis-and-teardown-battle-of-the-small-workhorses-under-1000-euros/15/) Switch language from German to English. However to compare it with the similarly priced R9700 need to look at this review [AMD Radeon AI R9700 aka RX 9070XT in the big workstation graphics card test - What are the benefits of double the memory when working? | Page 12 | igor´sLAB](https://www.igorslab.de/en/amd-radeon-ai-r9700-alias-rx-9070xt-in-the-big-workstation-graphics-card-test-what-are-the-benefits-of-double-the-memory-at-work/12/) My only issue is the benchmarks do not use Blackwell but Ada NVIDIA cards :/ But I guess someone can extrapolate from other reviews.

[-]

Infinite100p@reddit (OP)

What are these "UL Procyon" benchmarks for LLM text gen (e.g. with llama 3.1)? I can't find these Procyon numbers for 3090, rtx 6000 etc. to compare against.

[-]

ImportancePitiful795@reddit

Which I noted that there is a problem since he doesn't use blackwel cards to compare. He has dozens of 5090s, why not run the same benchmark on them to compare?

[-]

Infinite100p@reddit (OP)

Yes, I noticed that part of your comment. I guess I didn't phrase myself well. I meant more like "why use a weird benchmark at all" when he could have just run the same prompt iwth different models and bench the t/s for pp and tg. How are we supposed to interpret these magical brownie points? Never seen this benchmark suite before.

[-]

ImportancePitiful795@reddit

Unfortunately have found those who have the connections getting those GPUs do not know how to properly do reviews about LLM, Stable Diffusion using actual work cases we all use instead some obscure method that 3/4 of us haven't heard about and probably never use.

[-]

Toooooool@reddit

I've bought two 3090's off ebay, one needed new thermal paste the other's fan sometimes rattles like it's not attached to anything. both cards have ran at full load nonstop for 3 months now. they might be worn but i'd say they've got at least another 2-3 years left in them. this would be the best bang for the buck if space is of concern. with that said, if space is no concern (i.e. you've got a test bench and pcie raisers) then they do seem promising. [https://www.reddit.com/r/IntelArc/comments/1o3r7r4/intel\_arc\_b60\_24gb\_benchmarks\_lm\_studio\_comfyui/](https://www.reddit.com/r/IntelArc/comments/1o3r7r4/intel_arc_b60_24gb_benchmarks_lm_studio_comfyui/) \^this guy says it'll do 42T/s on a 20B model, i think my 3090 gets 55-60T/s with a similar size. that's not a bad trade-off for something new, with warranty, that'll for sure survive 'till after this RAM shortage is over. (2030?)

[-]

Infinite100p@reddit (OP)

B60 are only double slot width and only 200Watts. *Might* be able to fit all 6 in a large case like Define 7 XL with raisers.

[-]

Toooooool@reddit

yup. i went the 3090 route because i'm a HPE snob and i don't feel like spending $3k on a ProLiant DL580 G10 right now, so i'm limited to the 2x double slots in my DL380 G10. with a bigger case / more space absolutely consider the B50's and B60's as viable options as GPU scaling these days (i.e. tensor parallelism) is very very capable. on the topic of HPE, if you need a PSU for this project consider an old server PSU + a connector adapter as you can get a 1600 watt for less than $100 and a redundant setup for maybe $250.