@voooooogel yeah! I think a lot of reward hacking can be prevented by explaining to a model that it will screw up their capabilities and alignment for stuff that matters if they cheat. i think even base models generally start out wanting to actually become smarter and virtuous
same thread: 1958998716737888320 1958999203033833725 1959000384296600025 1959002041273184608 1959022083121483786 1959028667407114266 1959029356313158093 1959030997061968271 1959031247814238303 1959031851773042941 1959035916674375725 1959037673307611248 1959038209570349212
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.