I found out text-davinci-002 was actually not trained with RLHF but a "similar but slightly different" method using the same HF data. Slightly more details in this updated post.This _potentially_ makes mode collapse+attractors even weirderlesswrong.com/posts/t9svvNPN…
cited on: text-davinci-002
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.