LM Studio cant use my gpu as main

Posted by Zinxdia@reddit | LocalLLaMA | View on Reddit | 7 comments

as title says, lm studio always uses my cpu, I want to make lm uses the GPU tried several changes

Laptop specs

24gb ram
3070 8gb ram
i9-11 gen

i cant seem to use gpu as main resource for llama in lmstudio

[settings](

things I did
updated GPU
runtime engine is set to cuda 12 llama cpp
tried several changes in GPU offload, from maximum to half,
tried changes CPU thread pool
tried changes in context length

upon testing cpu usage spikes, while my GPU sits idle, only works if my cpu reach 70% above then gpu starts working

the models I used are :
open hermes 2.5 mistral 7b
OpenOrca Platypus2 13B q 4 k s
Mythomax L2 13b q4 k s

[-]

noage@reddit

You are using cuda 12 which needs i think a Blackwell x1 at least and then you can add on older cards. Use the standard cuda or even vulcan runtimes.

[-]

Zinxdia@reddit (OP)

its the same with the cuda 11, will test in vulkan later thanks

[-]

Rich_Repeat_22@reddit

Update the Cuda driver and switch to it. Dont use 12.

[-]

Creative_Bottle_3225@reddit

Nvidia CUDA 12.8 accelerated llama.cpp engine

[-]

Not sure why but I have found lmstudio has been bad at this for me too. I have found the default number of layers offloaded is usually too low and there is an overload setting I suspect to be a cause too. Koboldcpp was about 2x as fast for me. I keep lmstudio for low importance stuff since it can auto unload models and while slow never completely crashes for me. According to some other posts it's probably due to lmstudio using an out of date llamacpp version which is the root of lmstudio and koboldcpp.