Update on 2019 Mac Pro

Posted by habachilles@reddit | LocalLLaMA | View on Reddit | 18 comments

This post is for the haters it has best every one of my expectations so far. An amazing machine. Will update with more models. These are all
Small but it’s awesome.

[-]

egomarker@reddit

"2019 Mac Pro" doesn't tell us much about the machine's actual specifications. Also, what do you mean by "small"?

[-]

habachilles@reddit (OP)

Sorry this was in my last post. It’s 1.5tb ram with dual w6800x s 128gb vram

[-]

ParthProLegend@reddit

Wtf is that insane amount of ram. You might be a Ram billionaire in this economy.

[-]

habachilles@reddit (OP)

Yeah ddr4 but in still finding it super useful for this build.

[-]

ParthProLegend@reddit

Yeah, ram is very useful for pc. I upgraded 2-3 days before the ram crisis happened and man I am so grateful for that sweet 32gb I have now. Only had 16gb before and windows is shit

[-]

DistanceSolar1449@reddit

It tells enough- he’s running a Radeon Pro Vega II or possibly a Radeon VII

It’s basically the same chip as an AMD MI50, and gets roughly the same performance. These performance numbers look about right.

[-]

zqkb@reddit

Nice, what OS/Software stack do you use? I have one with 768gb ram + 2xVega II DUO, but not using it much. What would you recommend?

[-]

habachilles@reddit (OP)

Dude, I’ve been working so hard on at this past few days. I’m about to release some specs on Kimmy and GLM. Stay tuned for the next post but I’m running Lynx. macOS won’t do anything.

[-]

I clicked on this post to say I feel your pain as a 2019 MacBook Pro owner thinking this was going to be a "damn I shouldn't have bought that" post but that is actually great. Wild machine. How are the thermals/noise?

[-]

habachilles@reddit (OP)

Really good! I’m shocked. I just got kimi running. So far max 5tps but it’s the q4 and a beast.

[-]

disgruntledempanada@reddit

Get a screenshot of MacPow running (with the boring bits hidden lol). I wanna see how many watts that thing is chewing through. Prob shows all fan percentages too.

[-]

disgruntledempanada@reddit

Have Kimi build you a nice web gui for it as the prompt lol.

[-]

fzzzy@reddit

I agree. The 2019 Mac Pro is an AI beast. I have one for it.

[-]

habachilles@reddit (OP)

Let’s go!

[-]

libregrape@reddit

Happy for you! The numbers aren't really faster than my 5060 ti 16gb, but you can run large quants and models, thus making up in quality.

Though, your numbers for 27B dense look suspiciously low. Are you sure it's not misconfigured?

For reference:

  Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15839 MiB
| model                          |       size |     params | backend    | ngl | threads |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | --------------: | -------------------: |
| qwen35 27B Q3_K - Small        |  11.44 GiB |    26.90 B | CUDA       |  99 |      14 |           pp512 |       814.52 ± 19.15 |
| qwen35 27B Q3_K - Small        |  11.44 GiB |    26.90 B | CUDA       |  99 |      14 |           tg128 |         27.42 ± 0.14 |

[-]

habachilles@reddit (OP)

Dense was so weird! I don’t know I should look back over that I felt sus on it. And yes! It’s 5060 ish speeds with 128gb of vram!