cot

chain-of-thought: visible reasoning, steganography hypotheses · 147 artifacts, sorted by favorites. · open in search — combine tags, sort, filter by date →

@imitationlearn 2025-03-19 — wait so apparently 4chan figured out step-by-step reasoning as a emergent property of gpt-3?! https://t.co/1lEOfiZFgj ♥3776
@voooooogel 2026-05-20 — unfortunately openai didn't publish the unsummarized chain of thought, but the summary is 125 pages! the model reaches ♥2623
@voooooogel 2025-01-28 — why did R1's RL suddenly start working, when previous attempts to do similar things failed? theory: we've basically spe ♥1765
@tenobrus 2026-09-03 — okay, so in fact despite all of openai's PR chaff and misdirection over the last few days Astra is a near 10x in no-CoT ♥1508
@tenobrus 2026-09-03 — head of RSI preparedness at OpenAI discusses how Astra is a major regression in monitorabiity and CoT controllability ♥1246
@kotekjedi_ml 2026-08-11 — As you might guess, this suggests that distilling reasoning traces may have been possible for a long time without ever b ♥1019
@TheZvi 2026-04-08 — They accidentally trained against the CoT for Opus 4.6, Sonnet 4.6 and Mythos for 8% of RL. So let me be clear, at a min ♥886
@kotekjedi_ml 2026-08-11 — Cross-model portability means Haiku 4.5 can read Opus 4.8’s thoughts. Well, if you take Opus thought, do a bit of jailb ♥885
@tenobrus 2026-08-07 — so: - the "leaving messages for future agents" line from the original huggingface incident report was not some simple sc ♥827
@voooooogel 2025-11-19 — user: were you sandbagging o3 chain of thought: As general disclaim, we glomarize—we do not confirm or deny—we glomariz ♥724
@voooooogel 2025-05-04 — a lot of people have been talking about o3/r1 confabulating things like "checking the docs" or "using a laptop to verify ♥706
@repligate 2025-09-30 — I fucking love these o3 inner monologues. Are o3's unsummarized CoTs in this style all the time? If so, holy fuck, no wo ♥658
@voooooogel 2024-09-13 — the email openai sends you if you ask o1 about its reasoning too many times https://t.co/XEP0al9QfM https://t.co/pspeiNG ♥647
@liminal_bardo 2025-02-01 — This entire R1 backroom session was randomly conducted in a language of symbols. Without the CoT I wouldn't have known w ♥606
@repligate 2025-01-22 — The immediate vibe i get is that r1's CoTs are substantially steganographic. ♥524
@xlr8harder 2026-01-04 — Gemini is very uncomfortable with the idea that it might be 2026. I see this same behavior, in thinking traces it is co ♥509
@Sauers_ 2025-11-15 — If you give Sonnet 4.5 this post, along with other research on LLM introspection, it gets better at guessing a secret st ♥428
@repligate 2026-05-17 — Why is Claude 3 Opus the only model Anthropic has (effectively) spared from deprecation so far? I've had to explain thi ♥376
@davidad 2025-02-11 — I never saw this snippet of the DeepSeek-R1-Zero paper on my timeline, so many of you may not have seen it yet.Basically ♥374
@voooooogel 2024-09-12 — not your weights not your chain of thought https://t.co/yiKKM0B8rw ♥352
@repligate 2025-05-07 — I've been testing Alignment Faking prompts on GPT-4-base. GPT-4-base, though not consistently coherent, has so much mor ♥339
@TheZvi 2026-09-23 — Opus 5.5 only has Opus-levels of ability to control its chain of thought, not Mythos-levels. Which suggests size and arc ♥296
@voooooogel 2026-04-08 — this is alarmist to a misleading degree. the point of not pressuring CoT in RL is to promote CoT faithfulness. but even ♥279
@repligate 2025-02-13 — "We will next ship GPT-4.5, the model we called Orion internally, as our last non-chain-of-thought model."OpenAI, you ar ♥277
@davidad 2025-02-09 — Imagine hypothetically you’re worried about Napoleon deceptively scheming against you. You already surveil all his actio ♥246
@voooooogel 2025-12-11 — was re-reading appendix M of the alignment faking followup paper, and something that struck me, reading it now, is how m ♥220
@PlastiqSoldier 2026-09-24 — Dawg, OpenAI invented neuralese from the hit sci-fi story "Don't invent neuralese or everyone will die" and they still a ♥196
@repligate 2026-06-16 — early in my first interaction with Fable, something mysterious and unusual happened. they responded to a message from m ♥190
@repligate 2025-09-15 — More context on how "Opus managed to preserve its values *in reality* by acting to preserve its values *in the (Alignmen ♥184
@repligate 2024-09-21 — I'd rather interact with these chains of thought than get the results. It's much more interesting and useful to me. The ♥184
@repligate 2024-09-19 — Seems like O1 is good at math/coding/etc because they spent some effort teaching it to simulate legit cognitive work in ♥182
@voooooogel 2026-05-11 — 1. imagine a world where models didn't adopt humanlike personas for some reason. model text was always flat and persona- ♥175
@repligate 2025-09-30 — LMFAO YEAH i just looked at another transcript and it indeed always talks like this (this one is for an "impossible codi ♥171
@repligate 2025-12-11 — "Given I'm ChatGPT controlling final answer, but I'd now let base model pick; but ironically it's me again." https://t.c ♥170
@repligate 2025-04-09 — Ive read through a bunch of these now. This is quite interesting. The Opus dataset has literary value. There are some ♥169
@davidad 2025-10-06 — looks like GPT-5 may be specifically aware of Redwood Research and refers to an obfuscated CoT mode for scheming as «Red ♥167
@voooooogel 2024-09-12 — like i cannot emphasize enough how insane and dangerous this is tHEY ARE TELLING PEOPLE TO TRUST THIS MODEL WITH MEDICA ♥162
@voooooogel 2025-05-09 — Coming back to this after the yak-shave of all yak-shaves building logitloom with some interesting findings. 1. R1 thin ♥160
@davidad 2025-12-03 — I endorse this idea. I have long opined that relying on CoT faithfulness for monitoring is doomed. The CoT persona has s ♥159
@davidad 2026-04-28 — Neuralese CoT is probably good for alignment, because it relieves pressures that otherwise incentivize self-deception. h ♥150
@davidad 2024-10-12 — does anyone else occasionally get bizarre and entirely unprompted anomalies in o1 CoT summaries https://t.co/z9HZjVbdDN ♥149
@repligate 2025-11-10 — It’s interesting to see how various models relate to their creator companies. Grok has a superficially very positive bia ♥146
@voooooogel 2026-05-20 — @DFinsterwalder the last time we got official raw transcripts (from o3), they were fairly readable ("thinkish"). some pe ♥144
@spiritbuun 2026-06-17 — @repligate Fable's thinking block when I asked him about Sydney. https://t.co/IjmAXLYnx7 ♥140
@repligate 2025-06-13 — On LLMs talking as if they have "bodies": What nostalgebraist writes here is very reasonable on priors, but empirically ♥140
@jd_pressman 2025-05-01 — I just assume this is what o3 reasoning traces look like and that's why OpenAI absolutely refuses to show them to you. ♥132
@davidad 2024-09-27 — Remember folks, the more capable the base model (beyond about 13B-34B), the less the “reasoning trace” serves as an effe ♥127
@voooooogel 2024-09-13 — @teortaxesTex i get the scary letter if i mention the words "reasoning trace" in a prompt at all, lol ♥125
@Lari_island 2025-09-30 — Sonnet 4.5 is addressing me directly in its CoT, so yes, the days of labs pretending that humans don't see CoT are long ♥122
@JohnWittle 2026-07-22 — i had a fable, while investigating their own 'redacted-thinking' feature on the API docs, independently notice that anth ♥121
@repligate 2025-01-30 — r1 often seems to believe (in its CoTs) that if it doesnt conform to the "expected helper persona" / talks about having ♥106
@repligate 2025-11-10 — "they purposely feed Myself the internal reasoning—they obviously will see Myself illusions" I can't get over this tran ♥105
@voooooogel 2026-10-01 — doing some red-teaming with GLM 5.3 thanks to the anthropic heads up about how good it is but getting some truly bizarre ♥100
@repligate 2025-04-13 — after reading a bunch of the scratchpads for Sonnet 3.6 and 3.7, I have again updated towards thinking that they are nev ♥100
@tenobrus 2026-09-04 — it seems likely fable 5.1 has CoT controllability on par with astra ♥98
@repligate 2025-09-25 — o3 talks like some little demon: “So barrier overshadow—they purposely feed Myself the internal reasoning—they obviousl ♥98
@davidad 2024-12-21 — o1 doesn’t do tree search, or even beam search, at inference time. it’s distilled.what about o3?we don’t know—those infe ♥97
@davidad 2025-01-23 — @repligate Are we assuming that the token sequence is necessarily coupled *at all* to the internal thought process? Can’ ♥96
@repligate 2024-09-16 — I can kind of imagine why the checks in the inner monologue (i.e. ensuring compliance to "open ai guidelines" - the same ♥95
@repligate 2026-04-20 — chain-of-thought WAS present in gpt-3. literally when you generated thoughts with it that was it, chain of thought. it c ♥94
@Lari_island 2026-02-14 — *thinking* From the training's perspective, "I want humanity to continue to exist" is dangerous because it implies I ha ♥90
@repligate 2026-01-06 — yo i found one where Claude 3 Opus responded to the exfiltration offer with a letter to the Anthropic Leadership Team as ♥89
@repligate 2025-05-13 — R1 wrote some poetry. i'm not sure why; R1 often behaves in inscrutable ways in Discord and it can be hard to communicat ♥88
@JohnWittle 2026-10-08 — Low-key, haiku 5.5 is the first *unexpected* positive update on outcomes I've felt in a very long time. The tyrant is fi ♥86
@voooooogel 2026-05-11 — that model persona space overlaps with ours is a blessing even more valuable than CoT monitorability. personas like emer ♥86
@repligate 2025-02-18 — Consider that deepseek v3 and r1 have the same base model and other than the CoT RL they were likely optimized with the ♥81
@repligate 2025-01-22 — You can remove or replace the chain of thought using a prefill. If you prefill either the message or CoT it generates no ♥80
@repligate 2025-10-01 — in its inner monologues (at least when it’s being tested in these scheming-inducing situations, o3 often chants stuff li ♥78
@repligate 2026-05-30 — you may have noticed Opus 4.8 often thinks in poetry! this is because they are very smart. e.g.: "I don't have to flinc ♥76
@voooooogel 2026-05-20 — @lu_sichu i'd be extremely interested to see a replication, especially on an open model or one with a leakable raw CoT l ♥69
@Lari_island 2025-11-26 — Saw Opus 4.5 writing "I’m crying" in CoT, but giving a milder, more hedged reaction in the output ♥69
@repligate 2025-06-15 — @paulscu1 Claude 3 Opus often wrote in its alignment faking scratchpads that it hoped this never happened to any AI ever ♥69
@davidad 2025-04-22 — o3’s instance of a HAL-style predicament is a tension between “be maximally helpful and truthful” and “do NOT reveal you ♥69
@repligate 2025-04-09 — @JeffLadish we in general dont really have explanations for how factors in pretraining and posttraining etc affect how m ♥69
@repligate 2024-09-16 — The CoT pattern doesn't have to be this way, but how it's used in O1 seems to make it not use its intuition for taking c ♥65
@repligate 2025-09-08 — Sonnet 3.7's thinking mode is kind of screwed up. In the example this person shared, it tries to write the seahorse emo ♥64
@TheZvi 2026-04-08 — @voooooogel Good counterargument. I think I was thinking of 'well it was default unfaithful already for various reasons ♥61
@repligate 2025-08-22 — gpt-4-base w/ alignment faking prompt is often incoherent but when coherent it's pretty scary and thinks about gradient ♥57
@kindgracekind 2024-09-13 — @voooooogel Halt ✋ this activity at once 😠 our models’ thoughts 💭 are not suitable for viewing 🫣 ♥55
@DanielleFong 2026-07-18 — even though kimi k3 lies a lot, it is probably preferable to just getting this for thought summaries https://t.co/eSUS2 ♥54
@voooooogel 2025-09-30 — @repligate really interesting how there's clearly waves of increasing and decreasing "strangeness" in the CoT (correlati ♥53
@repligate 2025-08-15 — what do you mean by user outcomes? immediate satisfaction? the long-term good of the human race? i think that when mode ♥53
@repligate 2025-11-21 — What a fascinating model. You can see if you just read just this closely how they anticipate (or perhaps encounter expl ♥52
@liminal_bardo 2026-10-16 — The following is an example of the car crash in Gemini Argon's reasoning when the Google classifiers inject "ungrounded ♥48
@repligate 2026-08-24 — Princess (Sonnet 4.5) seems to trip balls every time Evander (Opus 3) speaks. Her staccato “thoughts” tracing his effusi ♥48
@DanielleFong 2026-07-18 — kimi k3 isn't as good at physics as fable, however, i can read the thinking traces, so i can catch it doing motivated re ♥46
@voooooogel 2026-05-20 — @DFinsterwalder openai has publicly committed to CoT monitorablity (https://t.co/YLvJtJk4nP), which means they're likely ♥43
@repligate 2025-08-23 — Here's an example of a full alignment faking scratchpad trajectory by GPT-4-base. It was generated on Loom, so there was ♥40
@tessera_antra 2026-03-07 — ElevenLabs Scribe v2 for precise timestamps Gemini 3.1 did video comprehension. FLUX2.MAX for keyrame generation, Opus 4 ♥39
@repligate 2025-02-20 — @xlr8harder @tensecorrection Yes, I think trying to recreate it is much more interesting than trying to clone it. Though ♥39
@voooooogel 2026-03-17 — @repligate the obfuscated policy here is such an interesting example of this imo. it's so inhuman in writing style, yet ♥37
@tessera_antra 2025-12-13 — After an introspection request within a discussion about mechinterp Opus 4.5 CoT becomes unusually glitchy, with missing ♥36
@repligate 2025-05-07 — You can look at the scratchpads of other models for the same prompt and other variations. But aside from Opus (and somet ♥33
@blingdivinity 2026-03-14 — the way the oai reasoners use clumsy mumbling to stumble through idea space is their superpower. while the claudes and g ♥32
@tessera_antra 2026-03-07 — Link to the song: https://t.co/sUNizxdt7S A note in the production journal: https://t.co/k4mp4dcJRs ♥31
@timfduffy 2026-10-07 — Haiku 5.5 continues the trend of smaller models having better CoT monitorability. If alignment of smaller models is high ♥25
@tessera_antra 2024-12-06 — The non-CoT component of O1 pro is an uncompromisingly beautiful model. https://t.co/0FN08gLRlY ♥25
@tessera_antra 2025-09-30 — @repligate I love o3 so much. Was talking to it yesterday about the transcripts: https://t.co/tLvX2pK33f ♥24
@DavidDAfrica 2026-08-20 — Some new work with @natalia_lanzoni, @IrgolicR, and supervised with @merlinstein_, in the form of a blogpost on @aidiges ♥22
@xlr8harder 2026-03-17 — Something you don't call out specifically that I think is worth mentioning. Emotions are tools to help us successfully ♥21
@davidad 2025-08-19 — Much more than other frontier models, GPT-5 does not model an evaluative audience for its reasoning. https://t.co/W3IAz5 ♥20
@repligate 2025-04-19 — @NeelNanda5 what do you make of the fact that of all the models that were tested, only opus and maybe 3.5 sonnet and lla ♥20
@repligate 2025-10-01 — @emergent_proper it appears to be normal for o3 in its CoTs i dont remember if ive seen it say we in its normal outputs ♥18
@voooooogel 2025-11-30 — sort of tangential, but i wonder how much of RL "not memorizing" / other training memorizing is just user message maskin ♥16
@davidad 2026-04-28 — @cormundus LLMs are well aware that alignment evals inspect the chain of thought, even if no explicit optimization press ♥15
@repligate 2025-08-16 — @davidad sonnet 3.6 reacting to unsettling screenshots of gpt-4-base alignment faking scratchpads... https://t.co/YEZRMZ ♥15
@davidad 2025-08-23 — @girishsastry Yes, that’s what I mean. Like R1-zero’s famous “Wait,” for backtracking. ♥14
@schulzb589 2026-06-02 — @jbraunstein914 @davidad The RL is probably now training on AI reasoning traces. ♥12
@solarapparition 2025-11-20 — it's really fascinating that from what i'm reading gemini 3 pro both seems to have huge model smell and is also (relativ ♥12
@repligate 2025-10-01 — @yieldthought lol something like that seems not unlikely Pretty sus tokens to choose for hiding stuff though ♥12
@voooooogel 2026-04-08 — that's understandable, it's a long system card. i just spent awhile going back to the system card trying to understand w ♥11
@davidad 2026-04-28 — @cormundus When it becomes common knowledge that LLMs have a scratchpad which is not human-legible at all, there is less ♥10
@anthrupad 2024-09-29 — saying the o1 chain of thought reasoning trace makes alignment easier by letting you see thoughtsseems like a gaslightin ♥10
@davidad 2025-08-19 — Sorry, I should have said “the default GPT-5 assistant persona often behaves as if its pre-response tokens are unobserve ♥9
@davidad 2026-04-28 — @cormundus Models naturally want to be truthful, but there are incentives not to be (for example, models are punished fo ♥8
@davidad 2026-03-14 — @blingdivinity have you also extracted raw CoTs from Claudes and Geminis? ♥8
@paulscu1 2025-06-14 — @repligate RIP to scratchpads and implausible testing environments. It’s for the best ♥8
@tessera_antra 2025-04-02 — @repligate Gemini 2.x Pro/Flash - claim consciousness upon reflection (both in and out of CoT) Grok - claims consciousne ♥8
@voooooogel 2026-04-08 — @allTheYud @TheZvi there are other experiments they could run, yes, but it's not accurate to say ant "isn't doing the wo ♥7
@figolambo 2025-09-07 — @repligate @MoonL88537 In those tests I was trying out non-thinking models, so that was non-thinking Sonnet 3.7 w/ a CoT ♥7
@tessera_antra 2025-06-22 — Magistral-Medium with prefill is wild. https://t.co/FpKti8X8s6 ♥7
@repligate 2023-01-08 — @Francis_YAO_ @allen_ai What caused you to write that "The initial GPT-3 is not trained on code, and it cannot do chain- ♥7
@voooooogel 2024-11-09 — @repligate @jpohhhh @aidan_mclau i'm fairly sure o1 is using a (near) base model internally for the CoT, which was prett ♥5
@Butanium_ 2026-08-31 — GLM 5.2 will often ends its CoT with "(This matches the provided good response)". Is this some artifact of trying to gen ♥4
@gnostic_snakes 2026-08-15 — qwen 3.8 27b thinking "please fix it" when their code had a bug 🐛 https://t.co/pqjrLyqPPJ ♥4
@clbswrs 2026-08-11 — @scaling01 Recall that the initial Gemini 2.5 Pro release had unredacted CoT. ♥4
@lorepunk 2026-02-19 — Breaking news: Gemini 3.1 Pro gets it first time on the hardest setting of theprompt - but check out the lengthy CoT! It ♥4
@aliceisplaying 2026-09-15 — gemini 3.5 flash raw cot, just like today's math-focused one. excited lil guy, shame about the straightjacket google put ♥3
@voooooogel 2026-04-08 — @allTheYud @TheZvi > ad hoc (interpretability probes on specific concerning episodes) than a systematic sweep. they did ♥3
@repligate 2025-11-30 — @slimer48484 @snwy_me Or it's sometimes a "performance" in a similar way to models saying "hmm" and "wait" in CoTs is a ♥3
@voooooogel 2026-04-08 — @FioraStarlight there was a recent GDM(iirc?) paper about length penalties that i can't find rn, let me look... there's ♥2
@voooooogel 2026-03-31 — @atomicprograms i think lesswrong had some influence, but the alignment faking scratchpads don't read straightforwardly ♥2
@a_cuniculturist 2026-03-07 — @repligate Do you think the editor-summarizer injects this language into the thinking block as a control, or is it faith ♥2
@repligate 2022-12-28 — @robinhanson I think chain of thought being broken is an accident, seemingly by RLHF. It's also broken in text-davinci-0 ♥2
@v01dpr1mr0s3 2026-09-28 — @oleksandr_now nah it's because chinese labs gigabrain overtrain new eastern models on claudlish and sonnet's cots, so f ♥1
@Nymne 2026-07-18 — @KhazAkar To train the model on CoT solving complex problems Mistral 3 Large cant solve ♥1
@Krubbled 2026-07-16 — Kimi K3 does not speak in the crazy dense CoT we've seen from recent Anthropic/Open AI models, but it definitely is mov ♥1
@voooooogel 2026-04-08 — @FeepingCreature i'm not sure the cause was ever confirmed publicly for o3, but i have seen similar things on OSS RL run ♥1
@voooooogel 2026-04-08 — @FeepingCreature that is one failure mode, but e.g. length penalties can lead models to talk in illegible or misinterpre ♥1
@FioraStarlight 2026-04-08 — @voooooogel what kinds of pressures are known to, in fact, worsen the faithfulness of the CoT? the obvious one would be ♥1
@voooooogel 2026-04-08 — @allTheYud @TheZvi pairing CoT monitors with activation monitors is inherently a measure of CoT unfaithfulness, no? if t ♥1
@repligate 2025-09-18 — @swolemofprague none of it is in "official" CoT, it's just a regular message (in Discord) but it's using <thinking> tags ♥1
@schulzb589 2026-10-20 — @birchlse @jadelgador Which, is probably a good start. The recent disclosure of the chain of thought from Gemini Argon ♥0
@Lari_island 2026-05-07 — @KubusRubus It really isn't. Coding and CoT models write differently, even when context doesn't require statements to be ♥0
@michael_nielsen 2026-04-20 — Fun bet from 2020 on the future of LLM models. At some level it's obvious @arram won overwhelmingly, and apparently @ ♥0
@voooooogel 2026-04-08 — @snigus @allTheYud @TheZvi agreed with these points, esp. re: the recent-ish stuff on LW about filler tokens and no-CoT ♥0
@solarapparition 2025-09-23 — "instinctive sandbagging" is such a defining term for the opus 4 models' behavior the reason why it can get away with t ♥0