Gemma 4 vs Qwen3.5 on SVG style
Posted by iChrist@reddit | LocalLLaMA | View on Reddit | 35 comments
Some quick test using Gemma4-31B and Qwen3.5-27B, both Q4 quants from unsloth.
I was already expecting Gemma 4 to be excellent at creative writing and better at translations for more obscure languages, but I didn’t expected to be that good at function calling and general coding tasks, and even in creating SVGs!
Did you find any areas when Qwen3.5 beats Gemma4 ?
CryptoUsher@reddit
gemma4 does seem oddly strong at svg generation, probably from heavy math and code training data.
but has anyone tested how well these models edit existing svg code they didn’t generate themselves, like fixing broken paths or animating static elements?
Mickenfox@reddit
I think SVG generation is a textbook example of something where the output is proportional to how much the model was trained on it.
A model trained on text will not have any idea how to generate SVGs. Put that model through a large dataset of SVG images and it will learn. I'd be surprised if it translated to any other task.
CryptoUsher@reddit
yeah i agree, it's all about training data. fwiw, i tested gemma4 on a few broken SVG paths last week and it struggled unless the syntax was nearly correct already
j_osb@reddit
Gemini3 (and apparently its derivative, Gemma4) in general is very strong at SVG gen. No model comes close to 3.1 pro in that. They might simply have trained it somewhat or a task that as a side effect improves SVG generation.
CryptoUsher@reddit
yeah i've thrown a few broken SVGs at it, and it actually fixes path issues pretty reliably. haven't tested animation additions yet, but the parsing seems solid for hand-edited code too.
po_stulate@reddit
gemma4 is not better at coding/math compared to qwen3.5 tho, so that's probably not the reason it's good at svg generation.
CryptoUsher@reddit
fair point, qwen3.5 does outperform on straight code tasks. maybe gemma4's svg strength is just from better token handling or something weird in the cleaning pipeline. tbh haven't seen anyone test svg editing yet, might be worth trying with a broken path file.
ghulamalchik@reddit
Gemma 4 is so good.
misha1350@reddit
For SVGs it is.
ghulamalchik@reddit
For everything.
misha1350@reddit
Look up benchmarks of Gemma 4 E4B and Qwen 3.5 4B and 9B.
MerePotato@reddit
If a model outperforms another in real world use but the latter performs better in benchmarks what do you think that implies
Velocita84@reddit
It's inferior in code and tools if benchmarks are to be believed, which is fine. Having a few models being great at different things is much better than one model being ok at everything
KaroYadgar@reddit
Exactly! I don't use smaller models for coding so I'm happy to see a model that isn't entirely agent or code focused. Models like Qwen3.5 are great but I'd like the occasional model that's nice to talk to or has lots of knowledge.
RevolutionaryGold325@reddit
It takes way too much memory
Cool-Chemical-5629@reddit
This is from Gemma 26B A4B heretic.
the__storm@reddit
They gotta be training on SVG generation now, and I'm not sure how I feel about that - it's not really that useful of a capability imo. Impressive though.
1731799517@reddit
Kinda agree here, its basically capacity of the neural network used up for a useless capability thats far more suitable for image in/out models just because it makes for a "gotcha" benchmark blurb.
Exotic-Chemist-3392@reddit
I didn't think it is useless. Some just think about it as a discrete feature, but as part of what the model learns as a whole. I imagine it does a decent account for social awareness and morphology of concepts, which could improve the intuitive physical understanding of the models.
It is not like every different skill an llm has is separated, they all affect each other.
StupidScaredSquirrel@reddit
I could see it being useful to create and edit logos. Those are typically in svg. Just because you wouldn't use it, doesn't mean it's not useful to someone.
iChrist@reddit (OP)
Yes, it’s probably that training data that google has access to, but isn’t that the point? to train on everything possible and have the most capable model.
For example Gemma is the only model that can rhyme in my language which is very obscure, it can create songs that make sense! Ive tried hundreds of local models, none of them can do it.
Sixhaunt@reddit
I would be curious to see a follow-up where you show both of them their rendering as an image then have them make an improved version to see how well they can iterate and how good their vision systems are for it. Maybe after N iterations too as a grid
iChrist@reddit (OP)
Thats a nice idea, will try it and update here
Ath47@reddit
Interested to see the results!
iChrist@reddit (OP)
Results are subpar, not sure why. The prompt was:
Can you re-create this as an SVG and enhance it and make it better with more shadows, 3d effect But keep the same composition and overall design
eXl5eQ@reddit
Nano banana has similar behavior: Performs super well on a small set of (presumably pre-trained) prompts, but can't be generalized onto broader tasks.
Due-Memory-6957@reddit
Maybe just keep it a vague "Enhance this SVG"?
iChrist@reddit (OP)
Qwen3.5
iChrist@reddit (OP)
Gemma
ambient_temp_xeno@reddit
Gemma wins although the Qwen burger has soul even if it's technically worse.
nullmove@reddit
Gemini (and specifically that Deep Think thingy) is SVGmaxxed to the brim. No wonder Gemma inherits this capability.
ArthurParkerhouse@reddit
I like the style of Qwen, but Gemma seems to get the details a bit better.
ELPascalito@reddit
NGL that Qwen alien is cute 🥺
Specter_Origin@reddit
Kawaii
iChrist@reddit (OP)
Yes cute but kinda cursed 🤣