# @voooooogel — 2023-09-11

♥91 ↻9 · https://x.com/voooooogel/status/1701290467185549729

Goes through designing a simple tokenization scheme, embeddings, the qkv weights and attention head, and projecting that back to get 100% accuracy at predicting a simple sequence! It even has explainers for matrices and softmax if you're a little rusty :-)
https://t.co/v3bB5DfrF0

tags: author:voooooogel, kind:tweet, thread-context, year:2023