# @voooooogel — 2025-01-05

♥371 ↻20 · https://x.com/voooooogel/status/1876044723791507883

high temp often gets (ab)used to "make models more creative," but it's really a hack, because logprobs conflate semantic unlikelihood with syntactic unlikelihood--on 405-base, "peeling the the( …)" is many OOMs more likely than "peeling the demonic (fruit)"i think the future… https://t.co/wW2dBLO2gt

tags: author:voooooogel, kind:tweet, model:llama-3-1-405b-base, on:llama-3-1-405b-base, year:2025
cited on: _dossiers/llama-3-1-405b-base.md, llama-3-1-405b-base