Now we don't have to update from the GPT-2 tokenizer for future models anymore. The anomalous tokens have become a mainstream sensation, and will appear many times in future train sets, finally paying rent to justify their place in the tokenizer vocabulary. Nice! https://t.co/rCmG1mFj7n
cited on: gpt-2
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.