koboldcpp-1.87.1: Merged Qwen2.5VL support! :)
Posted by Snail_Inference@reddit | LocalLLaMA | View on Reddit | 4 comments
Posted by Snail_Inference@reddit | LocalLLaMA | View on Reddit | 4 comments
David_Delaune@reddit
Is anybody able to get koboldcpp compiling? I'm getting an error: Not a name of any known instruction: 'movmatrix'
Looks like it's being caused by this movmatrix line. Looks like a bug to me, the movmatrix instuction is exclusive to Hopper. Adding a preprocessor check for CUDA_ARCH seems to fix it.
tengo_harambe@reddit
Only 7B and 32B? Doesn't mention 72B
formervoater2@reddit
samgreen/Qwen2.5-VL-72B-Instruct-GGUF has the quants and mmproj
BABA_yaaGa@reddit
Inference on video possible?