what are the limitations on the intel arc gpu?
Posted by dev_is_active@reddit | LocalLLaMA | View on Reddit | 7 comments
I'm looking at building a local AI rig, and I'm having a hard time sourcing GPUs I need,
I've noticed and been looking into these Intel ARC GPUs, but there seems to be a mixed sentiment around them.
I was looking for more input on why these would not be an ideal GPU to build on
Dave_from_the_navy@reddit
I have an ARC B70. They're excellent hardware (a little better in hardware compute and bandwidth than an RTX 4070 super. A lot better in VRAM), but you're purchasing it based on the promise that the software stack will mature over the next 3-9 months. Right now, I'm seeing about half the inference speed on the ARC B70 than on an RTX 4070 Super (using Qwen3.5-9B). Flash attention is broken on the SYCL (Intel's hardware translation layer) llama.cpp backend, making prompt ingest times about half the speed compared to the NVIDIA card, and taking up much more VRAM for the KV cache.
I have faith (perhaps misguided) that Intel will rapidly close this gap in the next year or so, so I'm still happy with my purchase. That said, caveat emptor if you're expecting perfection out of the box.
dev_is_active@reddit (OP)
Well I'm looking at buying 8 of them
How are they with pooling?
Dave_from_the_navy@reddit
My understanding is that pooling is supported, but be sure your have the PCIe slots and lanes to support it. I wish I could be more help, but I just have the 1 in my home server.
czktcx@reddit
Intel just want to push their own ecosystem, openvino/sycl/oneAPI/one blah blah, but people don't see benefits when there's no hardware.
VRAM is cheaper on Intel side, but need more developer to make it work well.
PermanentLiminality@reddit
They are not ideal due to the state of the software infrastructure that is required to do anything useful with them. Expect to spend a lot of time finding the combination that works. Expect some things to not work at all that would be trivial with Nvidia GPUs.
Everything is a tradeoff and there are plenty that you get with the cheap VRAM of a B70.
Not saying to not use them. Just know what you are getting into.
Hopefully Intel sends llama.cpp maintainers free cards. That will help a lot.
sn2006gy@reddit
Most people just want to go download CUDA stuff and not hack around. If you can use ARC and Intel tools and get your stuff to work, you can save a lot of money.
Ell2509@reddit
For sure. I am going the AMD route, with 9700 and w6800. Mostly plug and play with rocm, so far! I think the cuda moat might not be as deep as we think.