# @repligate — 2023-11-13

♥187 ↻7 · https://x.com/repligate/status/1723864725077782962

LLMs (at least GPT-3.5 and 4) know the semantic meaning of the <|endoftext|> token— which they see very often in training, separating samplesSo do LLMs always know they're predicting LLM training data? Do even base models have latent situational awareness? Kinda haunted by this https://t.co/JMmWbbfCey

tags: author:repligate, kind:tweet, model:gpt-3-5, on:observations, year:2023
cited on: observations