@ciphergoth yes, there are hundreds of layers between each token (*causally* between, though they are usually depicted as an orthogonal dimension of the architecture), half of which are literally called “self-attention layers” https://t.co/pNuaABv68u
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.