ROCm installation support on windows. HELP PLS.
Posted by CyBerDreadWing@reddit | LocalLLaMA | View on Reddit | 16 comments
I am really new to this process, and I recently did a cuda llama.cpp build on my 3060 mobile GPU, faced very less issues.
Now I wanted to utilize the VRAM of my main PC GPU which has amd gpu, 7900 gre.
I went away and installed HIP SDK from here:
Install HIP SDK — HIP SDK installation (Windows)
after that followed some github advise and reddit advise from official llama.cpp repo and Guide: build llama.cpp on windows with AMD GPUs, and using ROCm : r/LocalLLaMA
and
llama.cpp guide - Running LLMs locally, on any hardware, from scratch (this one is great for newbies)
installed llvm to provide openmp path as well.
after many iterations I came to this conclusion:
cmake --fresh -S . -B build -G Ninja `
-DCMAKE_BUILD_TYPE=Release `
-DCMAKE_INSTALL_PREFIX="C:\Users\dreadwing\AppData\Local\llama.cpp\ROCm" `
-DLLAMA_BUILD_TESTS=OFF `
-DLLAMA_BUILD_EXAMPLES=ON `
-DLLAMA_BUILD_SERVER=ON `
-DCURL_INCLUDE_DIR="G:/vcpkg/packages/curl_x64-windows/include" `
-DCURL_LIBRARY="G:/vcpkg/packages/curl_x64-windows/lib/libcurl.lib" `
-DGPU_TARGETS=gfx1100 `
-DGGML_HIP=ON `
-DCMAKE_C_COMPILER=clang `
-DCMAKE_CXX_COMPILER=clang++ `
-DOpenMP_C_FLAGS="-fopenmp -IC:/PROGRA~1/LLVM/include" `
-DOpenMP_CXX_FLAGS="-fopenmp -IC:/PROGRA~1/LLVM/include" `
-DOpenMP_C_LIB_NAMES="libomp" `
-DOpenMP_CXX_LIB_NAMES="libomp" `
-DOpenMP_libomp_LIBRARY="C:/PROGRA~1/LLVM/lib/libomp.lib"
And it gives me this output:
-- The C compiler identification is Clang 20.0.0 with GNU-like command-line
-- The CXX compiler identification is Clang 20.0.0 with GNU-like command-line
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/AMD/ROCm/6.4/bin/clang.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/AMD/ROCm/6.4/bin/clang++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMAKE_BUILD_TYPE=Release
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.51.2.windows.1")
-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: C:/Program Files/AMD/ROCm/6.4/bin/clang.exe
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - no
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: AMD64
-- GGML_SYSTEM_ARCH: x86
-- Including CPU backend
-- Found OpenMP_C: -fopenmp -IC:/PROGRA~1/LLVM/include (found version "5.1")
-- Found OpenMP_CXX: -fopenmp -IC:/PROGRA~1/LLVM/include (found version "5.1")
-- Found OpenMP: TRUE (found version "5.1")
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS - Success
-- HIP and hipBLAS found
-- Including HIP backend
-- ggml version: 0.9.4
-- ggml commit: 9eb9a1331
-- Found CURL: G:/vcpkg/packages/curl_x64-windows/lib/libcurl.lib (found version "8.17.0-DEV")
-- Configuring done (3.3s)
-- Generating done (0.2s)
-- Build files have been written to: G:/llama/llama.cpp/build
All is going well but as soon as I run the llama commands, the output is empty, no nothing nada,
PS G:\llama\llama.cpp> llama-cli.exe --help
PS G:\llama\llama.cpp> llama-batched.exe
PS G:\llama\llama.cpp> llama-bench.exe
PS G:\llama\llama.cpp>
something like this, nothing is printing.
I am running latest MSVC runtime and in visual studio 2022 I also installed latest msvc.
I think I am missing something really acute, can someone please help me in my findings?
Much appreciated, Thanks.
EDIT:
I did a standalone llama.cpp build that just works with CPU and guess what, that is also behaving in the same manner, now I am getting a little clueless, dependency is not getting resolved
CyBerDreadWing@reddit (OP)
Got the issue mafacka..
Explaination:
a bell in my mind rang as soon as u/deepspace_9 told me to look for curl dll in bin.
for further diagnosis I opened up "x64 Native tools command promt for vs 2022", yes thats the shitty name of this terminal if u have vs 2022 installed.
I ran
dumpbin /dependents ./llama-cli.exeAnd I got this output:
It is asking for libcurl.dll, which I searched in my path variables and it was not there.
So, I put this in path.
After that I have faced a new issue, that is zlib1.dll was not found, it is not shown in dependency tree but it is one hella requirement, and someone might go mad if this error is invisible to them,
So I just downloaded one from dll me, and scanned for viruses and then pasted it in my system32.
And finally, it worked!!!
Really appreciated your inputs u/cypher497 and u/deepspace_9. May u have a good day.
MeateaW@reddit
Honestly as someone that hasn't had to properly fight dll dependency hell this was a useful list of troubleshooting steps for me.
Having said that, I really don't like chucking random DLLs into system32, I would recommend if you are getting a DLL for a one off build/run of a system that you aren't 100% sure you want to live forever in your PC would be to place the DLL in the same folder as the executable.
Just as unsafe to execute it as before, but this time it is only available when the program you are testing is using it.
(Programs look for DLL dependencies first in the same directory as the executable before they go looking in the common locations and path variables)
cypher497@reddit
glad to hear you got it working!
Normally on windows when you run a program with a missing linked DLL it throw's up a popup saying something like 'The program can not run due to libcurl.dll was not found', seems strange that didn't happen, it also logs it in the System Event Log as an "Application Popup" with the exe name and missing DLL.
Dependency Walker is a great tool viewing the full nested DLL dependency graph that an EXE or DLL needs for future reference.
https://www.dependencywalker.com/
You can disable llama.cpp's need for libcurl if you do not need to download models using llama.cpp from HF, add "-DLLAMA_CURL=OFF"
CyBerDreadWing@reddit (OP)
Yup, It was really strange, so I investigated this as well, I tried to reproduce it by removing some environment variable and I was back to same no output.
It was somehow also not coming in event logs, so I opened up my x64 developer console of vs 2022.
In that when I ran the same llama-cli --help, I was able to see the missing dll errors, but not in my basic windows powershell, I don't know but it might be x64 vs x32 battle somewhere in my system.
And for the second part I needed curl binaries for llama that's why I did not used that flag.
Overall, it was a good experiment to remember.
deepspace_9@reddit
Try it in cmd not powershell, it could show you error message Did you copy curl dll to bin folder?
CyBerDreadWing@reddit (OP)
Thanks mate, this one was missing from my path I dont know how, my issue is resolved now, made my final comment above.
CyBerDreadWing@reddit (OP)
on my nvidia system, I also did not copy curl dll to bin but it worked. I will try it here if thats the case.
cypher497@reddit
you are only running the 'configure' stage of cmake (where it sets up the build for your system/compiler/etc into the 'build' folder), not the the build stage where it compiles and generates an executable.
if still in G:\llma\llama.cpp to run the build stage:
cmake --build build --config Release
CyBerDreadWing@reddit (OP)
Sorry I did not pasted that but I performed all the build steps.
cypher497@reddit
if you performed the build steps look in "G:\llama\llama.cpp\build\bin\Release" ( or "G:\llama\llama.cpp\build\bin\Debug" for debug builds) there should be llama-cli.exe
since "G:\llama\llama.cpp\build\bin\Release" is not going to be on your Terminal's search path, you can only run llama-cli.exe when you 'cd' into that folder and it is your current working directory.
cypher497@reddit
the other option if you can't get the compile to work (ROCm is not exactly easy) is to use the prebuilt versions from lemonade-sdk (assuming they support your GPU)
https://github.com/lemonade-sdk/llamacpp-rocm/releases
CyBerDreadWing@reddit (OP)
ok so I tried lemonade sdks and they work out of box thanks mate, since they are working that means my msvc vcredist is correctly configured and when I am building I am missing something which I need to find out, I am also going to compare lemonade sdk with mine, just to see what is getting missing after the build
CyBerDreadWing@reddit (OP)
I am going to go try this, but still my goal was to utilize hip sdk and create an automation for windows system to be ready to work on any gpu.
CyBerDreadWing@reddit (OP)
Yup tried that same result, after that I added that folder to path and restarted my system, but nothing works
Old_Box_5438@reddit
Maybe you're missing some environmental variables?
I compiled llamacpp recently (b6838 and b6949) for 7735hs 680m, tried it with official 6.4.2 and 7.9 rc1 they released for PRO 395. This is what I used in environmental variables Gui (under system variables) for compiling with 6.4:
HIP_PATH C:\Program Files\AMD\ROCm\6.4\
Path add "C:\Program Files\AMD\ROCm\6.4\bin" to the list already in there
having that last one in path is needed for llamacpp execution with rocm, as I think thats how it finds rocblas.dll.
this guide has some info on environmental variables you may potentially need, go to Ryzen APU-Windows-Tarball and it will have a config for 7.9rc1 build, \TheRock\build is roughly the same as \ROCm\6.4 for 6.4.2, though compiler paths are different, hip 6.4 sdk seems to have everything in \6.4\bin https://rocm.docs.amd.com/en/7.9.0-preview/install/rocm.html#install-rocm-zyj
Compiled it using a regular cmd, haven't really tried powershell or visual studio:
set HSA_OVERRIDE_GFX_VERSION="10.3.5" probably don't needed this one, but you can put your 11.0.0 set PATH=%HIP_PATH%\bin;%PATH%
cmake -S . -B build -G Ninja -DAMDGPU_TARGETS=gfx1035 -DGGML_HIP=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="C:\github\vcpkg\scripts\buildsystems\vcpkg.cmake"
replace the -DAMDGPU_TARGETS with the ones you need and it should build it as long as you have the kernels in ROCm\6.4\bin\rocblas\library. I ignored OpenMP warnings and it compiled just fine both for the CPU and rocm parts.
also, if llama-server.exe or others output blank, try to run it directly from file explorer, it may give you an error that way
fuutott@reddit
I don't know what the problem is, but I wish you luck, report back when you get this working.
Just from past experience compiling llama on linux the output seems super short.