Love these models
I always have fun with your MoE models. They even inspired me to make my own and post it to huggging face!
One issue I did have though, is that the chatml template appears to be alpaca. But some of the experts appear to use chatml. This is only an issue when using the chat completions api endpoint that apply a template for you though, and the model continues to run through its response, even though i see '<|im_end|>' sent in the text. When using text completions, and applying your own template, its not an issue, as i can specify this.
But neat models!
Interesting.
It is possible during fine tuning of the model(s) that "chatML" training slipped in (??)
Could also be an EOS on/off issue or some other sampling issue(s).
This model should be used with Llama 3 instruct and/or Command-R templates.
Alpaca should work okay too.