@norvid_studies a hypothetical from an ilya interview where a transformer is asked to predict the next token of a murder mystery novel after the detective says "and the killer is..."

to do this successfully it would need to reason over the plot, its world model, its own uncertainty, etc