If your DeepSeek Coder V2 is outputting Chinese - your template is probably wrong (as are the official Ollama templates)
Posted by sammcj@reddit | LocalLLaMA | View on Reddit | 14 comments
After a lot of messing around trying to figure out why DeepSeek Cover V2 was frequently outputting Chinese text and ignoring system prompts to only return English with the help of some nice folks on a Github issue I raised it has been found that the template being used by a lot of folks including the official Ollama template is incorrect.
The correct template should probably look like this for Ollama Modelfiles:
```
TEMPLATE """{{ if .System }}{{ .System }}
{{ end }}{{ if .Prompt }}User: {{ .Prompt }}
{{ end }}Assistant:{{ .Response }}<|end▁of▁sentence|>"""
PARAMETER stop "User:"
PARAMETER stop "Assistant:"
PARAMETER stop "<|end▁of▁sentence|>"
```
Note that the current official Ollama models for DSC-v2 have an erroneous extra space after `Assistant:` and are missing the eos token and stop word.
Discussion: https://github.com/deepseek-ai/DeepSeek-Coder-V2/issues/12#issuecomment-2181637976
14 Comments
wegwerfen@reddit
powerfuljack@reddit
kryptkpr@reddit
ab2377@reddit
sammcj@reddit (OP)
ab2377@reddit
Dudmaster@reddit
kirqeee@reddit
ihaag@reddit
sammcj@reddit (OP)
Account1893242379482@reddit
Aaaaaaaaaeeeee@reddit
urarthur@reddit
XMasterrrr@reddit