PSA: NVIDIA DGX Spark has terrible CUDA & software compatibility; and seems like a handheld gaming chip.
Posted by goldcakes@reddit | LocalLLaMA | View on Reddit | 129 comments
I've spent the past week experimenting with the DGX Spark and I am about to return it. While I had understood the memory bandwidth and performance limitations, I like the CUDA ecosystem and was willing to pay the premium. Unfortunately, my experiences have been quite poor, and I suspect this is actually handheld gaming scraps that NVIDIA rushed to turn into a product to compete with Apple and Strix Halo.
The biggest issue: DGX Spark is not datacentre Blackwell, it's not even gaming Blackwell, it has its own special snowflake sm121 architecture. A lot of software do not work with it, or [have been patched to run sm80](https://github.com/triton-lang/triton/issues/8335#issuecomment-3417643519) (Ampere, 6 years old!) codepaths which means it doesn't take advantage of blackwell optimisations.
When questioned about this on NVIDIA support forum, [an official NVIDIA representative said](https://forums.developer.nvidia.com/t/dgx-spark-sm121-software-support-is-severely-lacking-official-roadmap-needed/357663/9#p-1745639-h-1-when-will-sm121-receive-native-support-instead-of-sm80-fallbacks-10):
> sm80-class kernels can execute on DGX Spark because Tensor Core behavior is very similar, particularly for GEMM/MMAs (closer to the GeForce Ampere-style MMA model). **DGX Spark not has tcgen05 like jetson Thor or GB200, due die space with RT Cores and DLSS algorithm**
Excuse me?? The reason we're getting cut-down tensor cores (not real blackwell) is because of RT Cores and "DLSS algorithm"? This is an AI dev kit; why would I need RT Cores, and additionally how does DLSS come into play? This makes me think they tried to turn a gaming handheld GPU (which needs/supports unified memory) into a poor competitor for a market they weren't prepared for.
In addition, in the same post the rep posted what appears to be LLM hallucinations, mentioning issues have been fixed in version numbers and releases for software libraries that _do not exist_.
Just be careful when buying a DGX Spark. You are not really getting a modern CUDA experience. Yes, everything works fine if you pretend you only have an Amphere, but attempting to use any Blackwell features is an exercise in futility.
129 Comments
Comrade_Vodkin@reddit
IORelay@reddit
GifCo_2@reddit
Conscious-Secret-775@reddit
IORelay@reddit
ios-learner@reddit
EbbNorth7735@reddit
GifCo_2@reddit
EbbNorth7735@reddit
dtdisapointingresult@reddit
EbbNorth7735@reddit
dtdisapointingresult@reddit
FullOf_Bad_Ideas@reddit
EbbNorth7735@reddit
EbbNorth7735@reddit
GifCo_2@reddit
goldcakes@reddit (OP)
inaem@reddit
GifCo_2@reddit
PentagonUnpadded@reddit
muyuu@reddit
PentagonUnpadded@reddit
muyuu@reddit
PentagonUnpadded@reddit
muyuu@reddit
PentagonUnpadded@reddit
muyuu@reddit
RandoReddit72@reddit
broken_gage@reddit
Eugr@reddit
waiting_for_zban@reddit
Eugr@reddit
Caffdy@reddit
Eugr@reddit
No_Afternoon_4260@reddit
Eugr@reddit
No_Afternoon_4260@reddit
Eugr@reddit
No_Afternoon_4260@reddit
Eugr@reddit
No_Afternoon_4260@reddit
Eugr@reddit
funding__secured@reddit
Eugr@reddit
funding__secured@reddit
Caffdy@reddit
Eugr@reddit
conockrad@reddit
fallingdowndizzyvr@reddit
Eugr@reddit
waiting_for_zban@reddit
Eugr@reddit
LonelyConcentrate847@reddit
Caffdy@reddit
ThePrimeClock@reddit
Eugr@reddit
ThePrimeClock@reddit
SeymourBits@reddit
FlakyAd2685@reddit
eibrahim@reddit
Ok_Top9254@reddit
EvilPencil@reddit
Late-Assignment8482@reddit
goldcakes@reddit (OP)
Capable_Site_2891@reddit
Salty_Comedian@reddit
PentagonUnpadded@reddit
GoranjeWasHere@reddit
nopanolator@reddit
Historical-Internal3@reddit
No_Afternoon_4260@reddit
roxoholic@reddit
raiffuvar@reddit
Late-Assignment8482@reddit
txgsync@reddit
ozzie123@reddit
Front_Eagle739@reddit
AssistBorn4589@reddit
Ok_Warning2146@reddit
indicava@reddit
LizardViceroy@reddit
muyuu@reddit
Anarchaotic@reddit
muyuu@reddit
Cane_P@reddit
goldcakes@reddit (OP)
Cane_P@reddit
goldcakes@reddit (OP)
Cane_P@reddit
goldcakes@reddit (OP)
bajaenergy@reddit
ThePrimeClock@reddit
bajaenergy@reddit
ThePrimeClock@reddit
ThePrimeClock@reddit
Inevitable-Jury-6271@reddit
goldcakes@reddit (OP)
Inevitable-Jury-6271@reddit
FPham@reddit
roxoholic@reddit
FPham@reddit
Queasy-Direction-912@reddit
No_Strain_2140@reddit
pier4r@reddit
txgsync@reddit
Queasy-Direction-912@reddit
xor_2@reddit
koushd@reddit
No_Afternoon_4260@reddit
b3081a@reddit
koushd@reddit
Current_Ferret_4981@reddit
IulianHI@reddit
PhilippeEiffel@reddit
littlelowcougar@reddit
Front-Relief473@reddit
dtdisapointingresult@reddit
goldcakes@reddit (OP)
ayaromenok@reddit
jacek2023@reddit
BreizhNode@reddit
Dalethedefiler00769@reddit
Historical-Internal3@reddit
Serprotease@reddit
tr0picana@reddit
Historical-Internal3@reddit
Ok_Warning2146@reddit
Eugr@reddit
DesignerChemistry289@reddit