@Dorialexander @RiversHaveWings Here's a simple HuggingFace format LoRa you can play with to get a sense of how a decent RL tune compares to Mistral base. In my experiments it gives more coherent dialogue than the base model, has more interesting takes on stuff, with no mode collapse.

https://t.co/YmmgWNv8Lc