@StephenLCasper for all the criticisms RLHF gets from the alignment crowd, there's surprisingly not many papers/posts that compile the reasons together and provide a rigorous technical critique Janus' mode collapse post is good: https://t.co/bNxxhmLWmZ (though text-davinci-002 isn't rlhfd)
cited on: text-davinci-002
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.