# @cognitivetech_ — 2024-12-28

♥1 ↻0 · https://x.com/cognitivetech_/status/1873140462124712261

@voooooogel imaging what happens once the whole training corpus is meticulously refined!my impression is that pretraining now is largely raw data, and that's why a lot of heavy handed alignment is necessarybut as those datasets are continually refined, less alignment is needed later

tags: author:cognitivetech_, kind:tweet, thread-context, year:2024