@voooooogel i agree but am slightly suspicious that the non-confabulated prefill being out of distribution might account for the performance decrease. i wonder if you could look at the top logits and choose the top non-confabulation so that it’s in distribution?
in reply to: 1918876538193477956
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.