I think you should revisit the k:v cache quantization - it probably went dumb due to a combination of the model being below minimum viable context length + quantization... if you get get the context window size up, quantization's effects should lessen. Try:

 llama-server -m ~/models/unsloth/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf \                                                                                                              
    -c 131072 \                                                                                                                                                                  
    -ngl 99 \
    --flash-attn \                                                                                                                                                               
    --no-mmap \   
    --cache-type-k q4_0 \
    --cache-type-v q4_0 \
    --jinja \                                                                                                                                                                    
    --chat-template-kwargs 
'{"enable_thinking":false,"preserve_thinking":true}'
 \
    --host 0.0.0.0 --port 8080

flash-attn computes the attention matrix more efficiently so it's faster and uses much less VRAM, unlocking larger context lengths on smaller consumer systems. no-mmap forces loading the entire model at start, takes longer but once it is loaded it is faster, but most importantly on a smaller system it will give you an early warning if it is going to blow up. jinja is required for the template kwargs.

Dial back to -c 65535 if it still crashes. The quality hit on KV cache should be offset by giving it more context window.

Turning off enable_thinking helps in low-context environments. preserve_thinking is specific to Qwen 3.6 and keeps the models suppressed thinking tokens in the KV cache so it can still reference its own internal reasoning even though blocks aren't emitted in the output.

Also try a smaller quant, Q3_K_M drops from 22.1GB to 16.6GB and drops the model to less than half of your total memory, leaving more space for context. Agentic use like tool calling seems more tolerant of less capable models as long as it has the context window to orchestrate (At 32K context + opencode would get stuck in constant loops for me, 128K it runs non-stop and retries when it is too dumb to get it first time around).

I'm on a 20GB Ada 4000 and able to run this thing with 128K context without an OOM crash so far. It is the first time I've felt a local model be somewhat useful for agentic coding in terms of competency + inference speed... not replacing my Claude Max sub any time soon but it is actually usable for simple tasks and long-running jobs. I can even run it with the mmproj weights for multimodal if I offload a bunch of tensors to CPU. The memory accounting is a bit different with unified memory but can confirm that Qwen 3.6 seems to be a step up in terms of running on smaller memory systems, so there may be hope for you yet... good luck!