@davidad Even a conservative estimate of Alibaba benchmaxxing would falsify that. (e.g. Qwen2.5 dropped ~15pp OOD.) I think it's true with s/Sonnet 4/Sonnet 3 though.
in reply to: 2027075450481099153
same thread: 2027075450481099153 2027078289798443019 2027097260534993066 2027107533559591173 2027107671522922872 2027108925682635221 2027124649998299566
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.