@slLuxia @parafactual @tessera_antra @iyzebhel a good hypothesis that explains the tokenizer without a new basemodel and without terrible crimes against gods of performance https://t.co/d7HIWoJxxW
same thread: 2044475227975860328 2044487069633053066 2044487220699218281 2044525398885650722 2044626719555154079 2044884344784011659 2044884931185459218 2044929895537713200 2044984614809379118 2044985021929504859 2044986394314223683
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.