I’ve been testing Qwen3.6 27B MTP locally with llama.cpp on a dual RTX 3090 setup and wanted to share some numbers.

The main thing I wanted to compare was:

1. froggeric/Qwen3.6-27B-MTP-GGUF
2. Qwen3.6-27B-Q8_0-mtp.gguf
3. unsloth/Qwen3.6-27B-MTP-GGUF
4. Qwen3.6-27B-UD-Q8_K_XL.gguf

Both were tested with the same 32K context and the same prompt.

Hardware

CPU: Ryzen 9 5950X
RAM: 128 GB
GPU: 2× RTX 3090 24 GB
Total VRAM: ~48 GB
OS: Linux / WSL2 environment
Backend: llama.cpp CUDA

nvidia-smi / llama.cpp detects:

Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VRAM: 24575 MiB
Device 1: NVIDIA GeForce RTX 3090, compute capability 8.6, VRAM: 24575 MiB

llama.cpp builds used

For the regular MTP model I used a llama.cpp MTP build where the option is:

--spec-type mtp

For the am17an/llama.cpp mtp-clean branch, the equivalent option is:

--spec-type draft-mtp

So depending on the branch/build, the option name is different.

Model 1: froggeric Qwen3.6 27B Q8_0 MTP

Repo:

froggeric/Qwen3.6-27B-MTP-GGUF

Model file:

Qwen3.6-27B-Q8_0-mtp.gguf

Launch command:

./build/bin/llama-server \
  -m ~/models/gguf/Qwen3.6-27B-MTP-GGUF/Qwen3.6-27B-Q8_0-mtp.gguf \
  -a qwen36-27b-mtp-q8 \
  --host 0.0.0.0 \
  --port 8081 \
  -sm layer \
  -ts 1,1 \
  -ngl all \
  -c 32768 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  -b 64 \
  -ub 32 \
  -t 10 \
  -tb 20 \
  -np 1 \
  -fa on \
  -fit on \
  -fitt 1024 \
  --jinja \
  --reasoning off \
  --temp 0.7 \
  --top-p 0.8 \
  --top-k 20 \
  --spec-type mtp \
  --spec-draft-n-max 3

Test prompt:

Escribe una explicación técnica en español de unas 800 palabras sobre cómo MTP acelera la inferencia en modelos transformer.

Request:

curl -s --max-time 300 http://127.0.0.1:8081/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen36-27b-mtp-q8",
    "messages": [
      {
        "role": "user",
        "content": "Escribe una explicación técnica en español de unas 800 palabras sobre cómo MTP acelera la inferencia en modelos transformer."
      }
    ],
    "max_tokens": 1200,
    "temperature": 0.7,
    "top_p": 0.8,
    "top_k": 20
  }' | jq '.timings'

Result:

{
  "cache_n": 0,
  "prompt_n": 40,
  "prompt_ms": 545.415,
  "prompt_per_token_ms": 13.635375,
  "prompt_per_second": 73.33865038548629,
  "predicted_n": 1200,
  "predicted_ms": 27935.324,
  "predicted_per_token_ms": 23.279436666666665,
  "predicted_per_second": 42.956365925807766,
  "draft_n": 1527,
  "draft_n_accepted": 690
}

So roughly:

Generation speed: 42.96 tok/s
Draft tokens: 1527
Accepted draft tokens: 690
Acceptance rate: ~45.2%

Model 2: Unsloth Qwen3.6 27B UD-Q8_K_XL MTP

Repo:

unsloth/Qwen3.6-27B-MTP-GGUF

Model file:

Qwen3.6-27B-UD-Q8_K_XL.gguf

This was tested from the am17an/llama.cpp mtp-clean branch, where MTP is launched as:

--spec-type draft-mtp

Launch command:

./build/bin/llama-server \
  -m ~/models/gguf/Qwen3.6-27B-MTP-UD-Q8_K_XL-mmproj/Qwen3.6-27B-UD-Q8_K_XL.gguf \
  -a qwen36-27b-mtp-ud-q8-k-xl \
  --host 0.0.0.0 \
  --port 8081 \
  -sm layer \
  -ts 1,1 \
  -ngl all \
  -c 32768 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  -b 64 \
  -ub 32 \
  -t 10 \
  -tb 20 \
  -np 1 \
  -fa on \
  -fit on \
  -fitt 1024 \
  --jinja \
  --reasoning off \
  --temp 0.7 \
  --top-p 0.8 \
  --top-k 20 \
  --spec-type draft-mtp \
  --spec-draft-n-max 3

Same prompt, same 32K context.

Result:

{
  "cache_n": 0,
  "prompt_n": 41,
  "prompt_ms": 538.391,
  "prompt_per_token_ms": 13.131487804878049,
  "prompt_per_second": 76.15283316400163,
  "predicted_n": 1200,
  "predicted_ms": 51033.886,
  "predicted_per_token_ms": 42.528238333333334,
  "predicted_per_second": 23.513788465961618,
  "draft_n": 550,
  "draft_n_accepted": 540
}

So roughly:

Generation speed: 23.51 tok/s
Draft tokens: 550
Accepted draft tokens: 540
Acceptance rate: ~98.2%

Comparison

The surprising part is that the Unsloth UD-Q8_K_XL version had a much higher draft acceptance rate, but was still much slower overall.

In my setup:

42.96 / 23.51 = ~1.83× faster

So the Q8_0-mtp version was around 83% faster than the UD-Q8_K_XL version in this test.

My interpretation

The UD-Q8_K_XL model probably preserves quality better, but for MTP throughput it did not perform well on my setup.

The Q8_0-mtp model generated many more draft tokens, accepted fewer proportionally, but still achieved much better final throughput.

So for my use case:

Best for OpenCode / coding agent / interactive use:
froggeric Qwen3.6-27B-Q8_0-mtp.gguf

Best theoretical quality:
Unsloth UD-Q8_K_XL, but the speed penalty is large

Extra observation

I also tested the Q8_0-mtp model at larger context before, around 131K, and got roughly the same generation speed:

~43.20 tok/s at 131K
~42.96 tok/s at 32K

For this particular short prompt + 1200-token generation test, the max context setting did not noticeably affect generation speed.

Notes / caveats

• This is only one machine and one prompt.
• I did not do quality evaluation here, only throughput.
• MTP + vision / mmproj should be treated separately; I did not mix --mmproj with MTP for these tests.
• -np 1 is important for this MTP setup.
• Different llama.cpp MTP branches use different option names:
• some use --spec-type mtp
• am17an/llama.cpp:mtp-clean uses --spec-type draft-mtp

For now, my conclusion is simple: for a fast local coding model on 2× RTX 3090, Qwen3.6-27B-Q8_0-mtp is the winner.