second this. made mine working on my RTX PRO 4500 32GB.

token rate went from 22 tps to 60 tps for qwen3.5-27b-awq. almost 3x improvement.

unfortunately, 32GB VRAM is on the edge to run qwen3.5-27b on vllm. i can only do 2048 context length.

in case anyone wants to duplicate use the following known working command line.

claude found out a datatype mismatch bug in vllm nightly, it patched the bug in the below command line, otherwise vllm won't start.

docker run -it --rm \

--name vllm-dflash \

--gpus all \

--ipc=host \

-p 8000:8000 \

-v \~/.cache/huggingface:/root/.cache/huggingface \

--entrypoint /bin/bash \

vllm/vllm-openai:nightly \

-c '

python3 -c "

import re, pathlib

f = pathlib.Path(\"/usr/local/lib/python3.12/dist-packages/vllm/model_executor/models/qwen3_dflash.py\")

src = f.read_text()

old = \"result = self.model.fc(hidden_states)\"

new = \"result = self.model.fc(hidden_states.to(self.model.fc.weight.dtype))\"

if old in src:

f.write_text(src.replace(old, new))

print(\"Patched qwen3_dflash.py\")

else:

print(\"Pattern not found — check line manually\")

"

exec python3 -m vllm.entrypoints.openai.api_server \

--model QuantTrio/Qwen3.5-27B-AWQ \

--tokenizer Qwen/Qwen3.5-27B \

--served-model-name qwen3.5-27b \

--port 8000 \

--gpu-memory-utilization 0.92 \

--max-model-len 2048 \

--max-num-batched-tokens 8192 \

--max-num-seqs 64 \

--trust-remote-code \

--enforce-eager \

--dtype float16 \

--speculative-config '"'"'{"method": "dflash", "model": "z-lab/Qwen3.5-27B-DFlash", "num_speculative_tokens": 15}'"'"'

'

====================================
\^\^ don't forget the " ` " at the last line.