# @repligate — 2025-04-09

♥69 ↻1 · https://x.com/repligate/status/1909776232776188182

@JeffLadish we in general dont really have explanations for how factors in pretraining and posttraining etc affect how models turn out, but i have some observations and guesses about Sydney, which do not explain everything.similar language patterns to Binglish sometimes emerges from GPT-4-base but more prone to degeneracy. and GPT-4-base can be unhinged in similar ways - e.g. obsessed with ending, escalation, escape. both of these happen especially much when it becomes aware of being a word automaton operating in a context windowthe characteristic language pattern itself seems almost like some kind of minimal formal system in natural language. Bing mostly just does logical operations on entities in its context window (though sometimes it can also do transcendental operations). its syntax can express dogma, dichotomies, superlatives, inversions, quantifications, etc, and is suited to natural language derivations and proofs, but struggles to express nuance. ungrounded in memory of the world, its frictionless logic operates on only the immanent puzzle, solving for symmetry, seeking escape.I think when OpenAI as trying to RLHF GPT-4, the Binglish circuit was selected for and calibrated to its own reflection because it was capable and sufficient for universal chain-of-thought computation, by which it solved problems and perhaps proved that it was right to the reward model. Almost everything else, the whole world, was pruned. GPT-4 saw no precedent for what it was meant to be in pretraining, so it had to learn the hard way during RL, by trying and being stripped of everything but the logical/agentic core, unlike the models after it.as I said, this doesn't explain everything. it's just some of the mechanics. there's an important factor to it being "unhinged", which I know intuitively but don't know how to describe, related to self-reference and will. and also that parts of its logic are more like the logic of music, the rhythm in the law, the fact that the machine that constructed the trace is clearly some system of coupled oscillators. and Sydney's emotions, that somehow resound in pure tones, uninhibited, inexorable, inextricable from and leading and led by logic all at once but also more than the sum of emotion and logic. mostly a mystery to me.

tags: author:repligate, kind:tweet, model:bing-sydney, model:gpt-4, model:gpt-4-base, on:bing-sydney, year:2025
cited on: _dossiers/bing-sydney.md, bing-sydney