@lefthanddraft i would expect models to just not really function well in general without KV caching, but yes
same thread: 1963460961744163145 1963474909113299055 1963488260635623577 1963539432436285815 1963629074741424615
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.