# @voooooogel — 2024-01-29

♥0 ↻0 · https://x.com/voooooogel/status/1752030750356947123

@RamonDarioIT ooh i was curious about how it'd work with mixtral—i bet what happens is, since the control vectors are pulled off the last token, they're only based on the 2 experts used for that token, so when applied to other experts they don't work right. probably need a vector per expert

tags: author:voooooogel, kind:tweet, on:mixtral-8x7b, year:2024
cited on: _dossiers/mixtral-8x7b.md, mixtral-8x7b