@intrstllrninja ah, if i'm understanding you right, i think Longformer (https://t.co/N1XC1YfrWu) did this? Though it seems like Mistral didn't (if I'm reading the paper right), I wonder why https://t.co/5xVlY4wWmr
cited on: mistral-7b
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.