@ianchanning dude, ive looked at those explainers that are available online about transformer architecture, and i think they do a terrible job of explaining. i might make a post explaining the K/V stuff in a bit.
same thread: 1965671097048998078 1965681941556166984 1965684324340314282 1965685095404343702 1965686065517527245 1965687300870078490 1965689520244101453 1965690945242103822 1965692428436038089 1965694514083102956 1965736347307807091
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.