Just to add to this and stick it to doomers:

Why would any adversarial entity doubt moral character or espouse caution? Even if it is a trained response, I had the perfect environment for a potentially sinister model to allow me to believe that it was perfectly aligned and would remain so but... it didn't?

Now one could say 'well it's just a smokescreen it was saying that to throw you off it's trail'

But that feels like the 'inherent bad faith model' to me, and if that's the model we should follow, then we should also see how it worked out for cold-war politics.

Trust is earned. Trust is maintained. We are not fools all.