Help with MI50 and llama.cpp/ROCm 7.2

Posted by WhatererBlah555@reddit | LocalLLaMA | View on Reddit | 7 comments

I have an MI50 that I use with llama.cpp/Vulkan, however some models run quite slowly, so I'd like to try the ROCm backend, but no matter what I try it doesn't work. Downloading the missing files from ArchLinux package doesn't work. Downloading the files from coutryboycomputers doesn't work either Building RocBLAS also doesn't work. I'm not a Linux newbie, but at the same time I don't have days to spend on this issue... if someone has a guide that actually works to make llama.cpp run an MI50 with ROCm - ideally 7.2 - I would appreciate that.

Reply to Post

7 Comments

[-]

brahh85@reddit

this is the one i tried [https://www.reddit.com/r/LocalLLaMA/comments/1pkvc85/comment/ntysctk/?context=3](https://www.reddit.com/r/LocalLLaMA/comments/1pkvc85/comment/ntysctk/?context=3) this is another that should work [https://www.reddit.com/r/LocalLLaMA/comments/1s8thlo/build\_script\_for\_llamacpp\_for\_rocm\_including\_mi50/](https://www.reddit.com/r/LocalLLaMA/comments/1s8thlo/build_script_for_llamacpp_for_rocm_including_mi50/) this is a comparison of rocm and vulkan with mi50 [https://www.reddit.com/r/LocalLLaMA/comments/1rmt315/2x\_mi50\_32gb\_quant\_speed\_comparison\_version\_2/](https://www.reddit.com/r/LocalLLaMA/comments/1rmt315/2x_mi50_32gb_quant_speed_comparison_version_2/) what command are you using for llamacpp? what quant is your model?

[-]

WhatererBlah555@reddit (OP)

there are still a couple of things I need to figure out exactly, but this [https://www.reddit.com/r/LocalLLaMA/comments/1s8thlo/build\_script\_for\_llamacpp\_for\_rocm\_including\_mi50/](https://www.reddit.com/r/LocalLLaMA/comments/1s8thlo/build_script_for_llamacpp_for_rocm_including_mi50/) seems to work. Thanks.

[-]

juss-i@reddit

There are nightly build tarballs for the "preview" versions 7.12 and 7.13. AMD's version numbering is weird btw. Check out [https://github.com/ROCm/TheRock/blob/main/RELEASES.md#installing-from-tarballs](https://github.com/ROCm/TheRock/blob/main/RELEASES.md#installing-from-tarballs) and [https://rocm.nightlies.amd.com/tarball/](https://rocm.nightlies.amd.com/tarball/) (filter for gfx906). I grabbed one of the 7.12 nightlies a while ago, and it's been working fine. The only issue is that the tarball doesn't have an installer, so it's a bit more work to get all the binaries and libs in your path.

[-]

Material-Duck-6252@reddit

I have installed ROCm 6.3.3 (last official supported version for gfx906) and compiled llama.cpp on it. This served as a stable version for work. I would install ROCm 7.2, llama.cpp of ROCm 7.2 and vllm using docker for testing. \[Mixa's version\]([mixa3607/ML-gfx906: ML software (llama.cpp, ComfyUI, vLLM) builds for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60](https://github.com/mixa3607/ML-gfx906)) works well.

[-]