How are yall getting mistal nemo to work with llama.cpp?

Posted by Red_Redditor_Reddit@reddit | LocalLLaMA | View on Reddit | 3 comments

I get the following error when I try to run it: llama_model_load: error loading model: error loading model vocabulary: unknown pre-tokenizer type: 'mistral-bpe' I thought that perhaps llama.cpp needed some time to update and become compatable. I think others are running it, but I still havent been able to even with a fresh copy from git. Am I missing something here?

3 Comments

[-]

Iory1998@reddit

Update the software you are using. Once you get the latest llama.cpp, the problem will be resolved.

mrjackspade@reddit

I'm thinking the problem is your model, not llama.cpp The Nemo tokenizer is named "tekken" and not "mistral-bpe" Just a shot in the dark guess, I'm gonna assume that someone created the gguf you're using before support was officially added, and just guessed at the name of the tokenizer, since a lot of the other ones are *-bpe https://github.com/m18coppola/llama.cpp/blob/dd5a0bfffc9983258cc8827da4a476a907b853fe/convert_hf_to_gguf_update.py#L94

Red_Redditor_Reddit@reddit (OP)

You were right. I downloaded a new copy of the model and it works great. I really appreciate you.

Reply to Post

3 Comments