# @cis_female — 2024-06-20

♥9 ↻0 · https://x.com/cis_female/status/1803817236626292882

Not sure what 95% cache rate means here -- if i have a 20-turn conversation with the model where it keeps the kv cache in memory that's a 95% cache hit rate I think. MQA and local attention everywhere is interesting bc most people lose significant model capacity

tags: author:cis_female, kind:tweet, thread-context, year:2024