LLMs (at least GPT-3.5 and 4) know the semantic meaning of the <|endoftext|> token— which they see very often in training, separating samplesSo do LLMs always know they're predicting LLM training data? Do even base models have latent situational awareness? Kinda haunted by this https://t.co/JMmWbbfCey