If your DeepSeek Coder V2 is outputting Chinese - your template is probably wrong (as are the official Ollama templates)

Posted by sammcj@reddit | LocalLLaMA | View on Reddit | 14 comments

After a lot of messing around trying to figure out why DeepSeek Cover V2 was frequently outputting Chinese text and ignoring system prompts to only return English with the help of some nice folks on a Github issue I raised it has been found that the template being used by a lot of folks including the official Ollama template is incorrect. The correct template should probably look like this for Ollama Modelfiles: ``` TEMPLATE """{{ if .System }}{{ .System }} {{ end }}{{ if .Prompt }}User: {{ .Prompt }} {{ end }}Assistant:{{ .Response }}<|end▁of▁sentence|>""" PARAMETER stop "User:" PARAMETER stop "Assistant:" PARAMETER stop "<|end▁of▁sentence|>" ``` Note that the current official Ollama models for DSC-v2 have an erroneous extra space after `Assistant:` and are missing the eos token and stop word. Discussion: https://github.com/deepseek-ai/DeepSeek-Coder-V2/issues/12#issuecomment-2181637976