The first **Actual** Hebrew model: Zion_Alpha

Posted by Sicarius_The_First@reddit | LocalLLaMA | View on Reddit | 14 comments

I created it as POC about a year ago, but decided to finally upload it to HF: [https://huggingface.co/SicariusSicariiStuff/Zion\_Alpha](https://huggingface.co/SicariusSicariiStuff/Zion_Alpha) I saw different attempts by different people teaching a model Hebrew, but most had problematic tokenizers, and many times it would output garbage. This model actually knows Hebrew and retains all of its native abilities. I haven't fine-tuned it to do any task, including translation, however, it inherits all the abilities from the base mistral model it was fine-tuned from.  I tested it vs GPT3.5 at translation, and Zion\_Alpha won easily, then I tested it vs GPT4, which he barely, just barely also defeated. It shows that you can easily fine-tune a model to use a completely new language, without altering the tokenizer, even if that language is right to left, while the base model was pre-trained on a left-to-right language (English). Cheers, And have a good one.