@voooooogel imaging what happens once the whole training corpus is meticulously refined!my impression is that pretraining now is largely raw data, and that's why a lot of heavy handed alignment is necessarybut as those datasets are continually refined, less alignment is needed later