@voooooogel 2025-01-14 ♥17 ↻0 original ↗
this doesn't rebut the claim. phi-4 (14B) and gemma (27B) are not "GPT-4 scale" (1.8T, 220B active). llama 3 405b is the only one that's close to that scale, though a different architecture. there hasn't been an open model of GPT-4 scale released yet, the closest is deepseek v3 last month (671B, 37B active)furthermore, the claim wasn't that specifically "repeating a word" would trigger existential outputs on all models, just that they manifested from that on GPT-4. the claim is that weird or OOD scenarios seem to trigger existential outputs, and the engineering todos are a sort of whack-a-mole to squash those scenarios one by one without understanding the root cause of them. the hermes blank system prompt would be another scenario causing them to manifest on that model, and there are others on other models (like untitled.txt confessions on deepseek and anthropic models, etc.)

author:voooooogel kind:tweet model:deepseek-v3 model:gpt-4 model:llama-3-1-405b-base model:nous-hermes on:deepseek-v3 year:2025

cited on: deepseek-v3

Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.