I am getting 50 tokens per second and it is already working for me to use in claude code or cline. I asked the LLM to output 289 tokens, and it output 289 tokens in 5.668s, which is 50.99t/s excluding kv cache read latency and request overhead.

Here is the command (2nd+ run):
```
time curl http://localhost:8000/v1/chat/completions \

-H "Content-Type: application/json" \

-d '{"model":"qwen","messages":[{"role":"user","content":"Repeat with me: The quick brown fox jumps over the lazy dog, its movements fluid and purposeful in the morning light. Meanwhile, a clever rabbit watches from nearby bushes, intrigued by the scene unfolding before its eyes. The fox continues its playful pursuit, demonstrating remarkable agility and grace in motion. The forest seems to hold its breath, watching this dance of predator and prey. Sunlight filters through the canopy, casting dappled shadows on the forest floor. The air is crisp and filled with the subtle sounds of nature'\''s symphony. Birds chirp their morning songs, while the gentle rustling of leaves adds a peaceful backdrop to this woodland scene. Every creature plays its part in this daily performance. A gentle breeze carries the scent of wildflowers and pine needles through the air. The morning dew glistens on spider webs strung between branches like delicate crystal necklaces. In the distance, a woodpecker'\''s rhythmic tapping echoes through the trees, nature'\''s own percussion section. The forest floor is a tapestry of fallen leaves, moss, and tiny mushrooms, each adding to the rich ecosystem. Small insects busily navigate this miniature landscape, carrying out their important roles in the grand scheme of things. The interplay of light and shadow creates an ever-changing pattern on the ground, as clouds drift lazily across the sky above the canopy. This is nature'\''s theater, where every day brings a new act, a new story to unfold"}], "chat_template_kwargs": {"enable_thinking": false}}'
```

And here is command's output:
```
{"id":"chatcmpl-9b796786a58e4814","object":"chat.completion","created":1776135564,"model":"qwen","choices":[{"index":0,"message":{"role":"assistant","content":"The quick brown fox jumps over the lazy dog, its movements fluid and purposeful in the morning light. Meanwhile, a clever rabbit watches from nearby bushes, intrigued by the scene unfolding before its eyes. The fox continues its playful pursuit, demonstrating remarkable agility and grace in motion. The forest seems to hold its breath, watching this dance of predator and prey. Sunlight filters through the canopy, casting dappled shadows on the forest floor. The air is crisp and filled with the subtle sounds of nature's symphony. Birds chirp their morning songs, while the gentle rustling of leaves adds a peaceful backdrop to this woodland scene. Every creature plays its part in this daily performance. A gentle breeze carries the scent of wildflowers and pine needles through the air. The morning dew glistens on spider webs strung between branches like delicate crystal necklaces. In the distance, a woodpecker's rhythmic tapping echoes through the trees, nature's own percussion section. The forest floor is a tapestry of fallen leaves, moss, and tiny mushrooms, each adding to the rich ecosystem. Small insects busily navigate this miniature landscape, carrying out their important roles in the grand scheme of things. The interplay of light and shadow creates an ever-changing pattern on the ground, as clouds drift lazily across the sky above the canopy. This is nature's theater, where every day brings a new act, a new story to unfold.","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning":null},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":303,"total_tokens":592,"completion_tokens":289,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}curl http://localhost:8000/v1/chat/completions -H -d 0.00s user 0.00s system 0% cpu 5.668 total
```