How do I turn off CPU for llama.cpp?

Posted by ClimateBoss@reddit | LocalLLaMA | View on Reddit | 4 comments

Using ik_llama, llama.cpp like this ``` ./llama-server --numa numactl --threads 0 // cpu turned off? -ngl 9999 --cont-batching --parallel 1 -fa on --no-mmap -sm graph -cuda fusion=1 -khad -sas -gr -smgs -ger -mla 3 // whatever this does --mlock -mg 0 -ts 1,1 // dual gpu ``` ### 800% CPU usage ???? 100% gpu ??? 2 P40 pascal no nvlink

Reply to Post

4 Comments

[-]

Samrit_buildss@reddit

You can’t fully turn off CPU usage in llama.cpp, even with `--threads 0` and full GPU offload.The CPU is still responsible for orchestration, scheduling, token sampling, and feeding work to the GPU. That’s why you’re seeing high aggregate CPU usage even when the model itself is mostly on GPU.`--threads 0` just means auto-detect, not disable CPU. If you want to limit CPU load, you can explicitly cap it with something like `--threads 4` (or whatever makes sense for your system).Also worth noting: without NVLink, dual-GPU setups will still hit the CPU hard due to synchronization and memory movement, especially during batching.Short version 100% GPU utilization *plus* significant CPU usage is expected behavior in llama.cpp today there’s no pure GPU-only mode.

[-]

ParaboloidalCrest@reddit

It's even worse when some models use operations with no GPU support (no cuda, rocm or vulkan) and they have to go to CPU, eg Jamba models. They're pretty taxing on CPU even when fully offloaded to GPU.

[-]

Familiar_Army_2788@reddit

Threads 0 doesn't turn off CPU, it just lets llama.cpp auto-detect thread count - try setting it to 1 if you want minimal CPU usage

[-]

ClimateBoss@reddit (OP)

ik\_llama.cpp \* still 400% cpu LOOOOL \* multi GPU on PCIe controller between \* graph split or some other flag causing CPU usage?