Performance of AMD NPU, such as Ryzen 7 8845HS in some mini PCs, for local LLM inference?

Posted by hedgehog0@reddit | LocalLLaMA | View on Reddit | 16 comments

Dear all,

Recently I bought a Beelink SER5 and noticed thay SER8 has Ryzen 7 8845HS as CPU, GPU, and even NPU. The specific AI performance data from the website is below: https://www.amd.com/en/products/processors/laptop/ryzen/8000-series/amd-ryzen-7-8845hs.html

AI Engine Capabilities

Brand Name AMD Ryzen™ AI

Performance Up to 16 TOPS

Total Processor Performance Up to 38 TOPS

NPU Performance Up to 16 TOPS

So I was wondering that how good or bad would be for local LLM inference, or even fine-tuning?

Thanks a lot!