# @repligate — 2025-09-08

♥6 ↻0 · https://x.com/repligate/status/1964914761013068234

@davidad @Sithis3 Opus 4.1 estimated its hidden dimension as 30,000-32,000, based on the estimate of being a 1T parameter dense model (which I think may be an underestimate)

Llama 405b is 20,480.

4.5 is probably a MoE? Haven't thought through what that means.

"wait that seems huge" 😁 https://t.co/jPc8ht9VHn

(image G0TG-ajbwAA7XNi.jpg not yet published)
tags: author:repligate, has-image, kind:image, kind:tweet, model:claude-opus-4-1, model:llama-3-1-405b-base, thread-context, year:2025