@RamonDarioIT ooh i was curious about how it'd work with mixtral—i bet what happens is, since the control vectors are pulled off the last token, they're only based on the 2 experts used for that token, so when applied to other experts they don't work right. probably need a vector per expert
cited on: mixtral-8x7b
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.