Nvidia CMP 170HX for LLMs?

Posted by DeltaSqueezer@reddit | LocalLLaMA | View on Reddit | 21 comments

My GPU buying addiction leads me to look at weird and wonderful budget options. One option that came up was the Nvidia CMP 170HX - a mining card that is based on the A100 but cut down with only 8-10GB or VRAM but with a 1.5TB/s memory bandwidth. Unfortunately, Nvidia knew that people would want to use this and so nerfed it: https://niconiconi.neocities.org/tech-notes/nvidia-cmp-170hx-review/ It might still be possible to use, but given it isn't that cheap and you'd have to work around the various nerfing, I decided to skip this one.

Reply to Post

21 Comments

[-]

EducationalFishing56@reddit

Looks like ergo and Nvidia CMP 170HX is good match: [https://hashrate.no/coins/ERG/benchmarks](https://hashrate.no/coins/ERG/benchmarks)

[-]

Which-Client-224@reddit

Dead shit coin

[-]

MachineZer0@reddit

Someone conducted thorough research. CMP 170hx is a pass. The rest of the CMP line is usable. https://arxiv.org/pdf/2505.03782

[-]

MachineZer0@reddit

The CMP 70HX are nerfed too. Tops out at 110w on llama.cpp and exllamav2. TDP is rated at 220w. WhisperX tops out at 75w. The real kick to the nads is the fact that idle power draw is 65w.

[-]

Silly_Secretary_1440@reddit

so your saying you are using a 70 for llm? wasnt this whold decussion about how that doesnt work?

[-]

MachineZer0@reddit

Works fine for LLMs. Figured out idle power draw with nvidia-pstated.

[-]

a_beautiful_rhind@reddit

Chinese companies should buy them up and move them to a custom PCB with lots of vram. For regular users seems like a dead end.

[-]

DeltaSqueezer@reddit (OP)

Apparently the PCB is the same as the A100, all you need to do is put in the missing components. Though it isn't clear how much might have been fused off by the GPU ASIC. I know it is possible to restore 16x PCIe lanes, but it isn't clear whether you can flash BIOS to enable more VRAM or modify drivers to bypass FMA nerfs. I read of one group trying to bypass in drivers which surprised me as I expected this to be locked down in silicon.

[-]

a_beautiful_rhind@reddit

They lock a lot down in drivers and it's anyone's guess how much vram it really supports. On the 2080ti they just moved a resistor.

[-]

JakoDel@reddit

>On the 2080ti they just moved a resistor. kinda necro, but are you talking about the memory IC left empty compared to the titan or support for 2gb vram chips?

[-]

a_beautiful_rhind@reddit

I think they swapped the memory to larger chips.

[-]

fallingdowndizzyvr@reddit

> I know it is possible to restore 16x PCIe lanes Is it? I know people tried with the CMP 100HX and weren't able to do it. It was stuck at PCIe 1 x1.

[-]

DeltaSqueezer@reddit (OP)

The card is anyway x4 at the factory and apparently you can get to full x16 by adding the capacitors that were left off for the disabled PCIe lanes.

[-]

fallingdowndizzyvr@reddit

TBC, that would be PCIe 1 x16. And I think that speculation. Since by that link they don't say it can, they say "The PCB level lockdown **should** be trivially bypassable by hardware modification". Not that it has. People said the same about the CMP 100. Until people tried it by making the hardware mods. It was still stuck at x1.

[-]

DeltaSqueezer@reddit (OP)

From the screenshots, it also shows that the maximum PCIe version is 2.0 so I wonder if there is a way to get PCIe 2.0 x16 in an optimistic scenario. Anyway, I think for these cards it wouldn't make sense to gamble on that but use it for cases where the model fits entirely into VRAM or use in pipelined mode where the PCIe transfer rates is not a bottleneck.

[-]

fallingdowndizzyvr@reddit

> Anyway, I think for these cards it wouldn't make sense to gamble on that but use it for cases where the model fits entirely into VRAM or use in pipelined mode where the PCIe transfer rate is not decisive for performance. I don't think it needs to fit entirely into VRAM. You can use these for multi-gpu setups. You won't use it for tensor parallel, but PCIe 1 x4 is fine for layer splitting. PCIe 1 x4 ~= PCIe 3 x1.

[-]

DeltaSqueezer@reddit (OP)

>You can use these for multi-gpu setups. That's why I said: `in pipelined mode where the PCIe transfer rate is not decisive for performance`. Pipeline parallel is a particular way to set-up multi-GPU for throughput that doesn't require high interconnect speed.

[-]

ThisGonBHard@reddit

It uses HMB not GDDR from what I know. If the RAM is actually not on board, doubt you can add more.

[-]

DeltaSqueezer@reddit (OP)

Yes, it is HBM - so you are right, probably beyond the ability of a hobbyist to add more VRAM even if they could get hold of it.

[-]

DeltaSqueezer@reddit (OP)

It's crazy to think that at that time Nvidia had so many Ampere chips they were happy to burn them on these crippled miner cards than build more A100s.

[-]

Ill_Yam_9994@reddit

Sounds like the opposite of what I'd want in a LLM card. But maybe good for running a tiny model VERY FAST.