just added completion model (base model) support to logitloom, and it's really insane / depressing to see the difference between instruct and base model rollout diversity. left here is deepseek-v3 (no prompt, just prefill), right is 405-base.

even just from the number of branches you can see the base model is way more diverse--but look at the probability of the top first token: for deepseek it's 95%, but for 405-base it's only 3.35% (!!)

to sample 80% of possible first tokens with 405-base, you may have to explore HUNDREDS of branches, but with a chat model, maybe just one or two.