@atomicprograms @jcsemantics @Lari_island i dont even think it's only catastrophic forgetting; i think they probably generally rewind to earlier checkpoints when they make new versions and redo a significant amount of the posttraining
but yeah very well said