Qwen 30b a3b 2507 instruct as good as Gemma 3 27B!?

[-]

SlaveZelda@reddit

Well yeah thats true it great - but it's not multimodal like gemma.

Reply

[-]

Gemma3 is really, really great at text2text tasks, but it is so very bad at multimodal that I've been treating it like a text-only model. Qwen2.5-VL-72B-Instruct is still the only vision model I've used which was worth a damn.

Reply

[-]

randomqhacker@reddit

Wow, I've been super impressed with Gemma3's ability to OCR super messy handwritten notes including equations. What is it failing at for you?

Reply

[-]

ttkciar@reddit

It fails spectacularly at OCR'ing screenshots of hex dumps (doesn't even get the **number** of digits in a row right, let alone the right digits), and it slightly fails at identifying commodity networking hardware. It's a great model, and doesn't hallucinate much until it's used for vision. Then it hallucinates ***a lot!***

Reply

[-]

crantob@reddit

Tip from graphics guy, upscale your screenshots (x3) first. Often, frequency-based machines don't treat single pixel lines like your eyes do.

Reply

[-]

HomeBrewUser@reddit

GLM 4.1V 9B blows both out of the water

Reply

[-]

HumanAppointment5@reddit

How are you running the model? MLX and llama.cpp are not supporting it. Are you perhaps using vLLM or Transformers?

Reply

[-]

randomqhacker@reddit

My use case is handwritten OCR. Would you say GLM4.1V is better at that?

Reply

[-]

Tyme4Trouble@reddit

This is just my opinion here but I find Gemma 3 to be too sycophantic for use as a chatbot and I find it’s tool calling implementation very weak. For writing stuff (email drafting) I do like it. Maybe I need to spend more time tuning the system prompt, but every time I try to use Gemma 3 leave feeling more disappointed than anything. By comparison, I find Qwen-30B and Qwen-32B to be immensely more pleasant to use and generally smarter at that. I do miss vision capabilities and I can’t speak to its translation capabilities, but I just find myself enjoying Qwen3 more. Qwen3-Coder-30B is also very good for a GPU poor fella like me. OP if you’ve got any tips for getting Gemma not to sound like a over apologetic moron, please tell me your secret cause I’d really love a multimodal chat model that’s a better writer than Qwen.

Reply

[-]

Character_Stable_956@reddit

do you run it locally what setup ?

Reply

[-]

Tyme4Trouble@reddit

Locally I’m running vLLM with 128K context and to confirm it wasn’t some misconfig, I’ve also tested Googles API service.

Reply

[-]

jonasaba@reddit

No I mean also how much VRAM? What quantization?

Reply

[-]

Tyme4Trouble@reddit

48GB vRAM (2x 3090 TI or RTX 6000 Ada) W8A8 (RedHatAI) in vLLM or Q8_0 in Llama.cpp (Bartowski) / Ollama.

Reply

[-]

sersoniko@reddit

Do you know if it’s possible to easily disable the thinking of Qwen3 without having to write /no_think every message?

Reply

[-]

Tyme4Trouble@reddit

Yes I’m aware. I’m actually running Qwen3-30B-A3B-Instruct-2507 which doesn’t have thinking. They took it out because the hybrid reasoning functionality made the models dummer.

Reply

[-]

Tyme4Trouble@reddit

This is just my opinion here but I find Gemma 3 to be too sycophantic for use as a chatbot and I find it’s tool calling implementation very weak. For writing stuff (email drafting) I do like it. Maybe I need to spend more time tuning the system prompt, but every time I try to use Gemma 3, I leave feeling more disappointed than anything. By comparison, I find Qwen-30B and Qwen-32B to be immensely more pleasant to use and generally smarter at that. I do miss vision capabilities and I can’t speak to its translation capabilities, but I just find myself enjoying Qwen3 more. Qwen3-Coder-30B is also very good for a GPU poor fella like me. OP if you’ve got any tips for getting Gemma not to sound like a over apologetic moron, please tell me your secret cause I’d really love a multimodal chat model that’s a better writer than Qwen. Edit: Grammar

Reply

[-]

fp4guru@reddit

No. I would say coding , yes. Creative writing , not so good.

Reply

[-]

jadbox@reddit

What's the best model for creative writing do you think?

Reply

[-]

pitchblackfriday@reddit

Probably Gemma 27B?

Reply

[-]

H3g3m0n@reddit

I found the thinking model wouldn't stop thinking. It just kept commenting on what it was doing. I'm guessing it's because it was trained to use the 'thinking budget' that requires the engine to forcibly stop it in the middle of thinking but I'm not sure what implementations actually support that. I wonder if they could just train the model itself to do that, can they count tokens? For my case I ended up using the coder model, it's not trained on thinking but seems to do it if told it "You may also think by putting it between html block tags with the name 'think'. " That phrasing needed to be specific, I tried to tell it to use <think></think> tags but then it used <tool_call> (Possibly Open Webui rewrote it). It seemed to follow instructions fairly well, although now it's not thinking enough but I had a page of specific instructions that it mostly followed. Another problem I've seen is the models seem to often get stuck in a loop repeating the same output. Possibly when the context is full.

Reply

[-]

mpasila@reddit

Still worse world knowledge though especially on pop culture.. can't even answer this "who are the main characters of G5 My Little Pony?" (it sometimes figures out it's a thing but hallucinates the answer)

Reply

[-]

Kompicek@reddit

In my own experience is better by at least a mile :)

Reply

[-]

FabioTR@reddit

I find Gemma 27b much better for general knowledge, and for multilingual content. For other things they are on par. But the speed of qwen is impressive.

Reply

[-]

testuserpk@reddit

What gpu are you using, I want to test, I have 4060 8gb laptop and 3060 12gb in desktop. What t/s are you getting?

Reply

[-]

Hanthunius@reddit (OP)

I have a MacBook M3 Max 128GB of ram but I'm using the Q4 MLX version and getting 80/s+ on medium to long zero shot answers.

Reply

[-]

testuserpk@reddit

You have a beast, happy tokening.

Reply

[-]

mfarmemo@reddit

I'm shopping for that config. I want to test bigger models but only have the 18gb MBP M3

Reply

[-]

Hanthunius@reddit (OP)

It's great, you won't regret it.

Reply

[-]

National_Meeting_749@reddit

Call me part of the Qwen cult if you must, but Qwen 3 30a3b is a powerhouse model.

Reply

[-]

ElectronSpiderwort@reddit

On a well-defined problem, it does great. Fantastic even. It one-shotted a pygame program with a rotating triangle and a bouncing ball inside. I then tried it with a 4K prompt full of various history and conversational details, with the goal of responding as a participant in a conversation... It didn't do well in that task. My verdict: Worth the disk space, but not great at long, detailed requests

Reply

[-]

Klutzy-Snow8016@reddit

I actually find it and the thinking variant good for creative writing, unlike what people are saying here. I guess it's a matter of opinion.

Reply

[-]

xoexohexox@reddit

Yeah I don't get the hate for reasoning models for creative writing/RP I think they do a good job at preventing repetitiveness and moving the story forward. I think maybe people are just impatient.

Reply

[-]

LicensedTerrapin@reddit

I don't use it to write fiction, I use it as an editor. It does almost as well as GLM4 32b. Gemma is definitely good at fiction because it really doesn't just wanna be an editor. It wants to be the writer.

Reply

[-]

hedonihilistic@reddit

In your experience, are the glm models better overall or for specific tasks?

Reply

[-]

BlueSwordM@reddit

It's worse at writing stories than Gemma3, but Qwen3-30B-A3B-2507 just demolishes Gemma3 27B in all of my tests, while being far faster since I can offload the main MOE layers on GPU and do the rest on CPU.

Reply

[-]

toothpastespiders@reddit

I really like the thinking version, to the point where it's among the very few recent models that have felt significantly different than anything else of the same size and architecture. Seems like for a long time most of the models in this size range have felt pretty same'y. I haven't even bothered to run anything through my own benchmarks in ages because it became predictable and boring. But this has just been fun to play around with and explore its capabilities. It's pretty subjective, but it just feels significantly smarter than the original 30b release. The biggest thing for me is that it's pretty good at tool use and leveraging large amounts of RAG data. Not "great", but pretty good is still...pretty good.It's serviceable and runs on limited hardware, that's huge. I still haven't put it through my benchmarks, but in a way that's a compliment to the model. Messing around trying to best integrate tool use and RAG has been fun enough to keep me from locking up the GPU.

Reply

[-]