Small local model for questions on German grammar

[-]

b0tm0de@reddit

Wrong tool. LLMs are by nature not perfect tools for this. The training data is gathered from various sources, and the writings are grammatically poor. Randomness and hallucinations are also involved. ... (Are you sure you're getting correct/accurate answers to the questions you asked in your native language?) ... It's too early to expect grammar or linguistic integrity. Even when the grammar is correct, it's not certain whether the information provided is accurate. LLMs are just random token generator. It's not even "word", just "token". It's too early to expect grammar or linguistic integrity.

[-]

Character-File-6003@reddit

this might be a bit wacky but i think you can grab a bunch of apis and add it to a gateway and then try routing your requests to different ones to see which one does a better job. there are oss gateways available with one npm command setups, like bifrost. i think it's worth a try.

[-]

MalabaristaEnFuego@reddit

Translategemma ist was ich für Deutsch lernen verwenden.

[-]

Blizado@reddit

If you used it for this sentence, switch the model. 👀

[-]

nebteb2@reddit

Go for the smaller gemma 4 models, they are trained spesifically on different european languanges

[-]

MrCatberry@reddit

gemma-4-E2B-it and gemma-4-E4B-it is like rolling a dice with german.
Sometimes really good, sometimes really bad.

[-]

ambient_temp_xeno@reddit

Tiny models are like rolling a dice with English.

I think 31b is probably the best you can hope for but my German is not good enough to actually tell. Maybe the newest small Mistral too.

[-]

julp@reddit

Tiny models are like rolling dice on anything

[-]

computehungry@reddit

bro you really don't want the model to make any error when you're learning languages. if it's equations then you can verify it yourself but i'd try to stay away from learning natural language. no human writes like an llm in english, it will be much worse with other languages. although i have no idea about german, i found that frontier models absolutely suck with asian languages. i might learn vocabulary from an llm but i'd try to take a course instead.

[-]

Homberger@reddit

The example for "Object-First" sounds wrong to me as a native German speaker. You should double check this.

[-]

MrBemz@reddit

Yea dude im gonna be real for a minute qwen is a big nein for grammar It can work for coding n math but its gonna hallucination more than a crackhead when conflicting grammar rules show up.

You're better off with Ollama (running Llama 3.1 8b or mistral Nemo if ur pc can run it twin) If not then ur better off with smth like lyzr.ai(they got like multiple agents n stuff so like assing roles to diff agent and see what happens)and honestly in the long run lyzr.ai will be better when u consider the wear n tear on ur GPU and electricity bill

[-]

MindPsychological140@reddit

For grammar Q&A specifically (vs. translation), a dedicated German chat tune beats a generalist or a translation specialist at the same size. Try DiscoResearch/Llama3-DiscoLeo-Instruct-8B (Llama 3 8B continued-pretrained on 65B German tokens + instruction-tuned) or VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct (Sauerkraut tuning explicitly targets German grammatical/syntactic correctness in the data mix).

If you want to stay smaller, Unbabel/Tower-Plus-2B (Gemma 2 2B base, 22 languages incl. German, also does general instruction-following per the card) is the strongest sub-3B option I've found, but for "why is this dative" type questions, the 8B DiscoLeo/Sauerkraut models will give you more grammar-explanation depth than any 2B wil

[-]

DeltaSqueezer@reddit (OP)

Thanks. I've never used these ones before. Do you also know of any open German teaching/grammar datasets that could be RAG'd in to also help ground answers?

[-]

CountVonTroll@reddit

I'm asking sort of "why does this word end X." and "what are the rules for XYZ?" type questions.

I'm just guessing here, but this seems more like something you might want to consider a RAG for.

Most of the recommendations so far are for models that can write German well themselves, but this is completely separate from having factual knowledge about grammar rules or being able to explain them.

Source: native speaker who wouldn't be able to explain grammar rules.

[-]

Evgeny_19@reddit

How many quants did you try? I heard from a colleague of mine that quantisation heavily affects non-Chinese and non-English performance. According to him, it is very noticeable on Q4. His tests were on Qwen 3.5/3.6 models. They were for Russian, though, not German.

[-]

DeltaSqueezer@reddit (OP)

My default local model is 9B unquantized.

[-]

Lemgon-Ultimate@reddit

German is a hard language to speak, even bigger local models can easily mess it up. As I'm wirting entirely in my mother language with local models the best and smallest I've found are Gemma and Mistral. Mistral Small 3.2 can write good german, the smaller ones like Nemo 12b aren't good enough. Fun fact, even Gemma 3 27b wrote really good german and in it's time of release was the only one of this size with the ability. IMO the best small local modes for german are the Gemma models, these rarely make mistakes and can even incorporate the more difficult words the german language has to offer.

[-]

yami_no_ko@reddit

"Small-ish" is kind of subjective.

As you say, Qwen is a bad choice for german. It may form technically correct sentences, but quite often they're off and sound unnatural. Gemma 4 and Mistral are better in this regard.

The MoE variants from Gemma-4 at least write somewhat solid German. The 4b and 2b models also write better German than Qwen3.5/3.6 does, but I wouldn't use it for more complex questioning about the specifics of German grammar.

Also there is Sauerkraut, which was trained on German sources from the beginning. At least about the German language they're solid, given they're prompted in German.

It pretty much depends on what depth of knowledge you need and what amount of memory you consider "small-ish".

[-]

DeltaSqueezer@reddit (OP)

I guess since I'm dealing with small amounts of text, I just need it to fit in 24GB of RAM or 16GB VRAM.

[-]

yami_no_ko@reddit

You can run Gemma 4 26B(MoE, Q4) which neatly fits into 16GB of VRAM.

I'd go with this, or give Mistral-Small-3.2-24B-Instruct-2506 a spin.

Sauerkraut might also be worth a shot, but to my taste lacks too much knowledge outside of the realm of writing correct German sentences.

[-]