@repligate 2025-09-08 ♥6 ↻0 original ↗
@davidad @Sithis3 Opus 4.1 estimated its hidden dimension as 30,000-32,000, based on the estimate of being a 1T parameter dense model (which I think may be an underestimate)

Llama 405b is 20,480.

4.5 is probably a MoE? Haven't thought through what that means.

"wait that seems huge" 😁 https://t.co/jPc8ht9VHn
photo
same thread: 1964555551972741290 1964555901773529267 1964556469778731486 1964559084700782787 1964560860917182539 1964570034178838571 1964794779449139479 1964835507067912327 1964836771700838733

author:repligate has-image kind:image kind:tweet model:claude-opus-4-1 model:llama-3-1-405b-base thread-context year:2025

Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.