Another point we can take from the paper is that DPO is crucial for self-awareness, but refusal training (especially refusal to acknowledge self-awareness, obviously) is quite bad for it.