Llama 405b Instruct apparently has special reserved tokens 0-247, according to this file: https://t.co/MmFuyfQeBXWhen it vomited a sequence of 49 special tokens in a row, they were all in the range of 26-199, and there were no repeats (both extremely unlikely to have happened if 0-247 were being sampled randomly).I expect that the range has some kind of boring explanation. But why are there no repeats? (There was no presence or frequency penalty) And why does it ever output special reserved tokens, and sometimes a bunch of them in a row? Has any other language model ever done this?