Minisforum MS-R1 - ARM based Linux computer with 64GB RAM
Posted by Rick_06@reddit | LocalLLaMA | View on Reddit | 4 comments
Out of curiosity, what is the likelihood of being able to run a 30b class model in a Minisforum MS-R1, an ARM based Linux computer with 64GB RAM?
Here the specs: ARM CIX CP8180, 12C/12T, 2.6GHz, 28W TDP, 45 TOPS (NPU 28.8 TOPS), 64GB LPDDR5 5500MHz RAM
Formal-Exam-8767@reddit
I would expect bad PP performance and TG being what memory bandwidth allows.
FullstackSensei@reddit
Llama.cpp compiles perfectly fine for ARM, so not sure exactly what you mean by "likelihood" here. A more appropriate question is why would you want to do that? IIRC, it's still a dual channel system, so performance will be slower than anything running DDR5-5600, at best.
I see it's on sale now, which isn't a bad deal, but don't expect any miracles from it.
ImportancePitiful795@reddit
None at all.
CalligrapherFar7833@reddit
Do you see any proper native arm support in any of the popular ones llama.cpp or vllm ?