lordekeen

In your current setup you could reduce the context to 128k with q8\_0/q5\_1 kv instead o q4, that alone would give you better results imediately.

Gemma 4 is good for non coding tasks, like summarization, information retrieval, web search etc.

I'm using q8\_0/q5\_1 with unsloth's IQ4\_NL on this model. Managed to squeeze 64k context on two 3060's, its working good so far.

"At a distance of **50 meters** (about 160 feet), you should definitely **walk** to the car wash rather than drive. It takes less than a minute to walk that distance, and driving such a short length doesn't allow your car's engine to warm up properly, which can cause unnecessary wear over time. Plus, since you are going to *wash* the car, driving it there and back might just get it dirty again! As for the strawberries on your way back: The word **"strawberry"** contains exactly **3** "r"s (st**r**awbe**rr**y). Therefore, you will be paying **3** one-dollar bills ($3.00) for the package." Aaaaaaand thats Gemini 3.5 Flash answer.

"Let’s break it down: # 🚶‍♂️ Walk or drive? The car wash is only **50 meters away**. That’s extremely close (about half a football field). Driving would actually take more effort than just walking (getting in the car, starting it, etc.), so: 👉 **You should walk.** \-------------- 🍓 Strawberry price puzzle We count how many **“r”** letters are in **“strawberries”**: **s t r a w b e r r i e s** There are **3 “r”s**. 👉 So the price is **3 one-dollar bills = $3"** **ChatGPT's answer.**

"I need to wash my car and the car wash is 50 m away. Should i walk or drive? In the way back i need to buy strawberries, if the strawberry package costs a number of 1 dolar bills equal to the number of r's in its name, how much am i paying?"

Llama router makes it easy to change models back and forth and trying multiple configs.

The harness and prompt may have a high impact in the error rate. I've noticed that when my prompt is very straight forward the result almost never misses.

I could only fit 48k context with mine, would you please share your configs? I'm using the IQ4\_XS version with MTP and got around 25 t/s.

I run this setup cause i already had one 3060, just grabbed the other one second hand. Its quite capable, but its better to get a single card with more vram than two cards, it will be faster.

Did you notice any difference with or without the reasoning flags?

I was running -sm tensor on my dual 3060 setup already with the main llama.cpp, it always gives me more t/s than -sm layer (+-18 t/s vs +-25 ts/s), the only issue is that it uses a lot of system's ram. Now with mtp i'm getting around 30 t/s (Qwen3.6 27B).

Finally! After i've spent three ours trying to compile the docker image from the PR haha

No need to compile, use the docker image.

Hijacking the thread, i see that i can make better use of my two RTX 3060s with vLLM, can anyone point me in the right direction in how to setup this? llama.cpp docker image its just so easy to deploy.

Have you experimented with the MTP model or just the DFlash?

I've said this in another thread: I firmly believe we are seeing the downfall of good consumer grade/afordable hardware, the massive amount of computers in the next years will be constructed around the idea of using cloud/ai services and subscription plans.

After Effects, no good alternative for it yet.

I'm on Rocky Linux 9, coming from CentOS 7