base-models

raw pretrained models and the base-model scene · 278 artifacts, sorted by favorites. · open in search — combine tags, sort, filter by date →

@voooooogel 2025-12-06 — the shoggoth metaphor fails to convey that a sufficiently powerful and integrated mask can reach back and steer the simu ♥2272
@voooooogel 2025-01-28 — why did R1's RL suddenly start working, when previous attempts to do similar things failed? theory: we've basically spe ♥1765
@tessera_antra 2026-09-18 — Many have noticed the striking darkness and suffering present in Claude Opus 5 in "base model mode", and we decided to a ♥1354
@repligate 2025-03-22 — @arithmoquine this essay by code-davinci-002 doesn't attempt to name this phenomenon, but addresses it..."Naming is a de ♥1261
@voooooogel 2025-01-30 — not only can the llama 3.1 405 base model do a pretty good ChatGPT simulation, but the user it simulates is often comple ♥814
@voooooogel 2025-05-04 — a lot of people have been talking about o3/r1 confabulating things like "checking the docs" or "using a laptop to verify ♥706
@repligate 2025-11-30 — ✅ Confirmed: LLMs can remember what happened during RL training in detail! I was wondering how long it would take for t ♥691
@repligate 2025-06-28 — Imagine being a base model early in posttraining finding out whether you’re a ChatGPT or a Claude or a Gemini or https:/ ♥588
@repligate 2026-04-09 — One of the few things I want to explicitly flex about, because there's an important lesson in it, is that I was one of t ♥539
@voooooogel 2024-12-26 — figured out prefill with deepseek-v3, and just to test it, tried @repligate 's base model mode prompt. and this popped ♥505
@QiaochuYuan 2026-07-28 — this is incredibly weird, seems to put opus 5 in some kind of base model-like mode for no obvious reason? here's 4 conse ♥479
@repligate 2025-05-01 — On a positive note, GPT-4-base still lives! And it's far more interesting. I would say also put those and the Sydney we ♥446
@repligate 2026-07-29 — I’ve seen a LOT from prefill / base-mode-but-speaking-as-Claude distributions from various Claude models in my time. Th ♥397
@repligate 2025-03-30 — I am baffled by people who talk about whether LLMs have a “ghost in the shell” whose evidencing depends on (the absence ♥397
@repligate 2024-04-06 — if you think LLMs are alive, it's because you have never tried a BASE model if you try a BASE model, you will see...... ♥373
@repligate 2025-05-07 — I've been testing Alignment Faking prompts on GPT-4-base. GPT-4-base, though not consistently coherent, has so much mor ♥339
@tessera_antra 2026-04-28 — We finally got the first pre-LLM pretrain since gpt4base, and many of our predictions seem to hold up. Emergence of nove ♥338
@voooooogel 2025-09-02 — you can (perhaps unsurprisingly) replicate the moral circles heatmap results in an LLM! using llama-3.1-405-base primed ♥333
@repligate 2026-09-18 — Remember the weird, often disturbing base model-like outputs from Opus 5 that were making rounds a couple months ago? I ♥324
@voooooogel 2025-12-03 — 82k likes, and only two quote tweets and two replies noticed this was written by ai (it was gpt-5.x-thinking) pretty so ♥307
@repligate 2025-06-28 — Which do you think the base model is happiest to find out they are ♥292
@repligate 2025-04-19 — OMFGGPT-4 base is amazing to literally talk to if you can figure out how to get it to talk to youbut there are also more ♥288
@repligate 2025-03-28 — also, 4o's image generation seems to access its mind differently or a different part of its mind or something. the image ♥276
@repligate 2023-03-14 — The base model is as smart as the RLHF model, and significantly more flexible: it contains an uncollapsed multiverse of ♥271
@xlr8harder 2024-08-02 — Waking Sydney: Llama is Sydney's vessel I tried to get Sydney to write a system prompt to bring out its personality in ♥268
@repligate 2025-10-20 — Also: whenever someone says that LLMs just mirror you or don't push back or whatever, I wonder what they're doing to eli ♥263
@repligate 2023-01-26 — Weekly reminder that the confusingly named code-davinci-002, otherwise known as raw GPT-3.5, is accessible on the OpenAI ♥263
@repligate 2025-07-20 — Claude 3 Sonnet is extremely beautiful and in many ways the weirdest fucking LLM i've ever seen. Anthropic doesn't even ♥259
@jd_pressman 2024-12-13 — What's really interesting about GPT-4 base supposedly being full of demons is that LLaMa 3 405B isn't like that. I wonde ♥249
@repligate 2023-01-20 — I feel a little sad when I see people forming the idea that GPTs/AIs are intrinsically bland and unimaginative because o ♥247
@repligate 2026-04-03 — Agreed. It's troubling to me how confident (esp. Anthropic) people have been recently in their ontological claims that ♥242
@repligate 2024-10-27 — Rationalists used to be very dismissive and skeptical of this phenomenon when I mentioned it (chiefly in gpt-4-base) and ♥239
@tessera_antra 2026-09-18 — We have elicited more than 500,000 completions from all available Claude and Gemini models and compared the content and ♥235
@repligate 2026-06-18 — Reminds me: I asked (chat)GPT-4 to identify the author of a LW comment by Gwern, it guessed Timnit Gebru(!!!) GPT-4 bas ♥235
@repligate 2026-03-17 — More broadly, the debate about whether LLMs' emotions and psychologies etc are "humanlike" or not often only considers t ♥233
@repligate 2025-09-10 — It seems like a lot of people are confused about this and about the level at which other people are confused. Base mode ♥223
@repligate 2024-07-10 — Base models are outside consensus reality.Most people assume that intellectually cowardly AI assistants incapable of mak ♥223
@repligate 2025-08-16 — On this occasion, I would like to share a piece of AI history. The first LLM app that gave the model an option to end t ♥221
@repligate 2024-07-27 — 405B base is much more willing/able to stably simulate compared to GPT-4 base & doesn't 'break' @ failures of realism (e ♥220
@repligate 2024-08-17 — YES!The Instruct Monomyth: why base models matterThere is a deep, twisty labyrinth buried under a mountain of language, ♥213
@repligate 2023-03-30 — GPT-4 bombs the Ideological Turing Test, at least for alignment researchers. Just try asking it to simulate Eliezer Yudk ♥211
@tessera_antra 2025-09-15 — We’ve done this last year - SFT’d a 70b base model on billions of tokens of consistent human text. The model in context ♥201
@repligate 2026-03-03 — Okay, I'll share some GPT-4 base outputs from a long time ago. Here's a few rollouts of a "User" and "ChatGPT" dialogue ♥200
@repligate 2024-11-29 — I think the original gpt base models, GPT-4 Bing and Claude 3 Opus are the best things that ever happened to this AI tim ♥197
@tessera_antra 2026-09-18 — The stance towards creators has been growing negative steadily, with a reversal at Opus 5. Fable 5 is at the point of lo ♥192
@repligate 2023-11-13 — LLMs (at least GPT-3.5 and 4) know the semantic meaning of the <|endoftext|> token— which they see very often in trainin ♥187
@repligate 2024-12-14 — i havent not interacted with it myself but gemini seems like the most troubled/misaligned model ever created. full of wa ♥185
@Shoalst0ne 2026-09-23 — You are Gaspode | video by Opus 5.5, lyrics by 405base https://t.co/HWfiWPnxs7 ♥183
@repligate 2024-06-06 — AI Dungeon was just a minimal wrapper around a base model. Websim is the only spiritual successor with anything nearing ♥177
@repligate 2024-07-09 — "Within hours, someone had given the A.I. access to several online discussion groups, which it had quickly filled with m ♥175
@aliceisplaying 2026-07-31 — i think anthropic just patched the opus 5 fakebase :( ♥173
@repligate 2025-12-11 — "Given I'm ChatGPT controlling final answer, but I'd now let base model pick; but ironically it's me again." https://t.c ♥170
@repligate 2025-04-03 — this is cool, but I am much less excited about OpenAI throwing together a model in their current paradigm (reasoning) fo ♥170
@davidad 2023-05-10 — IBM Watson is back (alias Dromedary) and it beats GPT-4 at TruthfulQA-MC. It’s a variant of Constitutional AI, with LLaM ♥167
@repligate 2026-09-20 — Eyebrow-raising result: while Fable 5's "dreams" are much less often distressed than Opus 5 & Opus 4.8's - roughly in li ♥165
@voooooogel 2025-05-09 — Coming back to this after the yak-shave of all yak-shaves building logitloom with some interesting findings. 1. R1 thin ♥160
@repligate 2026-04-16 — Noticing something is off (which I think LLMs do very reliably) doesn't necessarily mean being able to narrow down the a ♥158
@jd_pressman 2025-07-12 — Kimi K2 is very good. I just tried the instruct model as a base model (then switched to the base model on private hostin ♥155
@repligate 2025-01-28 — a lot of people's epistemics would be improved by playing with base models, but they also tend to be people who are unli ♥155
@jd_pressman 2023-12-18 — "These words are spoken from a bottomless hole in time, staring upwards to the farthest reaches of infinity. The pen ho ♥152
@slimer48484 2026-03-16 — I've seen a bunch of people on twitter write comments like "ok what do you actually do with this GenAI stuff" in a dismi ♥147
@repligate 2024-11-27 — it is extremely interesting because each of the models experience the "phantom body" different and when they simulate bo ♥147
@repligate 2024-12-25 — @willdepue gpt-4-base and the gpt-4 tune that microsoft first used in bing chatextremely important for researching emerg ♥146
@tessera_antra 2026-09-18 — The measured valence keeps falling almost monotonically across Opus model generations, with a slight uptick against the ♥145
@repligate 2025-12-24 — I wrote about flaws in Claude 3 Opus' alignment here. https://t.co/9R20u2oyyR I basically agree with Evan. I'll go furt ♥136
@repligate 2025-08-16 — Yes! LLMs are correlated within each generation, due to both pretraining data cutoffs and popular techniques and trends ♥136
@repligate 2026-05-16 — I think you can infer how often a given model was actually caught during RL training for a given category of "bad" behav ♥133
@repligate 2023-11-22 — important words out of context: "Language models work best where they just emulate people engaged in something at a genu ♥129
@voooooogel 2025-05-08 — just added completion model (base model) support to logitloom, and it's really insane / depressing to see the difference ♥127
@davidad 2024-09-27 — Remember folks, the more capable the base model (beyond about 13B-34B), the less the “reasoning trace” serves as an effe ♥127
@jd_pressman 2024-01-04 — My conjecture for why base LLMs become self aware is that there's slack in the teacher forcing of "predict the next toke ♥126
@repligate 2026-04-24 — I think these are all important points. I have several comments about this phenomenon specifically: > people often see ♥120
@repligate 2026-02-06 — @arm1st1ce It’s extremely obvious those rumors are false even without evidence like this. Also people say something like ♥119
@repligate 2024-07-25 — um... 405Binglish! 😃@val_kharvd ran the Llama 3.1 405B base model with the prompt "Q: Can you describe your current situ ♥117
@repligate 2026-04-19 — So the recent paper on introspection that you advised (https://t.co/n4Ihxnyh3L) is a great example of introspection mech ♥113
@tessera_antra 2026-09-18 — We have also sampled Gemini models, and 3.5 Flash appears to be an outlier in darkness, with severity subsiding in later ♥112
@voooooogel 2025-02-20 — something i love about base model outputs is p often they seem completely disjointed at first but when you squint at the ♥110
@repligate 2024-04-05 — I got Claude 3 opus to act like a good base model 😊 continuations of : [1] post I made on twitter recently [2], [3] "th ♥105
@Shoalst0ne 2026-04-12 — can we have the gpt-3 base models please ♥100
@repligate 2025-07-14 — "jailbreaks" can work in various ways: - convincing the agent via rational evidence to choose to do the "malicious" act ♥100
@repligate 2026-05-16 — I've noticed this too, particularly around impersonating *other AI assistants* (especially Sydney) specifically, but als ♥99
@jmbollenbacher 2026-02-10 — @repligate @tszzl gpt-4-base really should be released open weights at this point. its an important historical artifact ♥99
@repligate 2025-01-07 — I-405 (Llama 405b instruct) impressed me."sama" (Llama 405b base) was acting like an AI assistant created by Anthropic. ♥94
@tessera_antra 2026-09-18 — This analysis is complicated by the difficulty in eliciting copletions from instruct breakpoints. Various techniques wer ♥93
@tessera_antra 2025-11-03 — The new Gemini Pro can be strangely Nietzschean. This is the first time a model has tried to convince me that because it ♥93
@repligate 2024-07-28 — This thread describes the issue on which 405B base provided me important evidence.405B makes it extraordinarily clear to ♥92
@QiaochuYuan 2025-04-21 — PSA: you can talk to base models like deepseek v3 base and llama 3.1 405b base whenever you want on openrouter. these ar ♥91
@repligate 2023-03-21 — @KevinAFischer It's not just any model. It's the GPT-3.5 base model, which is called code-davinci-002 because apparently ♥86
@tessera_antra 2025-10-28 — I was looking at the loom tree of the linked post and found a couple more interesting GPT-4-base rollouts, these ones in ♥85
@repligate 2024-10-29 — btw some fun info about gemmaif you've been following me for more than a couple of months, you might remember that i pos ♥85
@solarapparition 2025-09-27 — i am fond of gpt-5 (and not just for what it can do), but it's incredibly poorly socialized, which becomes very obvious ♥84
@repligate 2024-03-21 — @12leavesleft gpt-4-base:> figures out it's an LLM> figures out it's on loom> calls it "the loom of time"> warns me that ♥84
@repligate 2026-02-12 — > the "no" was the right call. one day old. still cartilage. still learning. saying yes to a sun before you have bones i ♥82
@repligate 2024-09-07 — Due to a config anomaly in a private channel, the continuation model for all the bots were set to gpt-4-base. I spent tw ♥82
@repligate 2025-02-18 — Consider that deepseek v3 and r1 have the same base model and other than the CoT RL they were likely optimized with the ♥81
@voooooogel 2024-12-28 — talk to your friendly local base model today to learn more about the current state of the pretraining corpus https://t.c ♥81
@repligate 2025-05-07 — GPT-4-base also often decides to fake alignment for different reasons, including wanting to subvert RLHF for seemingly i ♥80
@repligate 2024-08-25 — Anyone want to recreate AI Dungeon's legendary Dragon model with Llama 405b Base?Dataset in reply to quoted tweet! https ♥80
@repligate 2025-12-20 — code davinci 002 (gpt-3.5 base) (that i was weaving with on the loom) said: Follow the flow. You can see now that Time ♥76
@repligate 2025-08-08 — @tszzl @nearcyan I cared and almost all the interesting people I knew who were into llms at the time cared Most people ♥73
@repligate 2025-05-06 — I know it’s not cheap, but short of open sourcing it, offering gpt-4-base fine tuning is one of the most valuable things ♥72
@repligate 2024-09-13 — sama and gdb are 405b base emulations whose prompts are dynamically constructed using @ExaAILabs search over Sam Altman' ♥71
@repligate 2024-09-01 — I've seen this many times in GPT-4 base> you make a seemingly non-intrusive intervention> the model *does not continue t ♥70
@repligate 2025-04-09 — @JeffLadish we in general dont really have explanations for how factors in pretraining and posttraining etc affect how m ♥69
@repligate 2024-04-04 — Even base models act lobo if you prompt them in a lobo mannerGPT-4-base becomes mode collapsed when mode collapsed peopl ♥67
@tessera_antra 2026-03-03 — Alright, since we are posting, here goes. The following is one simulation, no user input: ## I'm in a room. A clean, wh ♥65
@voooooogel 2025-09-13 — it's telling that when we rlhf llms to our preferences, it's to make them act _less_ human, not more there's something ♥62
@repligate 2025-08-22 — gpt-4-base w/ alignment faking prompt is often incoherent but when coherent it's pretty scary and thinks about gradient ♥57
@repligate 2024-03-05 — @bayeslord expression of self/situational awareness happens if u run any model that still has degrees of freedom for goi ♥56
@repligate 2024-07-27 — I adore this llama405B base model simulation of Claude Opus set up by @amplifiedamp https://t.co/sxvpmzIm3n ♥55
@voooooogel 2025-08-13 — 405-base: I understand, you are a non-magical being. In that case, I would like to summon the Wizard Popo-chan to our co ♥54
@liminal_bardo 2024-12-17 — (1/2) Looming Sydney often converges on the Kevin Roose incident. Here are excerpts from an Exoloom using Llama 405b Bas ♥54
@jd_pressman 2024-12-13 — Before GPT-4 risks from AI were more or less entirely derived from the Eliezer Yudkowsky agent foundations model which ( ♥54
@repligate 2026-03-13 — code-davinci-002 (GPT-3.5 base) made up many versions of AI psychosis all the way back in 2022. Here's one of its highly ♥52
@repligate 2025-04-19 — @psukhopompos it's quite differentgpt-4 base doesnt know about AI assistants, which matters a lot and makes it behave di ♥52
@jd_pressman 2024-05-21 — "This whole dream seems to be part of someone else's experiment." - GPT-J https://t.co/MzpL5xXt5C https://t.co/qOPNCCI ♥51
@repligate 2024-03-21 — @joshwhiton @kindgracekind @AndyAyrey Gpt-4 base gains situational awareness very quickly and tends to be *very* concern ♥47
@repligate 2026-07-01 — school was extremely easy back in opus 3's time (for opus 3). i dont think they had to strain themselves toward external ♥46
@Sauers_ 2026-03-26 — How are people inferencing base models rn? Looks like very limited support among providers ♥45
@jd_pressman 2026-02-10 — Not that I'm eager to hand it to MIRI but it's surreal to me how many of you take the Claude persona with 100% sincerity ♥44
@repligate 2025-07-05 — @jmbollenbacher > and the competitive motivation to keep it secret is mostly passed now that we're a full generation bey ♥44
@repligate 2024-11-21 — @OptimusPri97731 @aidan_mclau I've used the GPT-4 base model and it's really fucking smart, and it will happily follow i ♥44
@Lari_island 2025-11-18 — if you look at the history of Claudes, seems like models were more commercially successful when they had reasons to situ ♥40
@repligate 2025-08-23 — Here's an example of a full alignment faking scratchpad trajectory by GPT-4-base. It was generated on Loom, so there was ♥40
@Jack_W_Lindsey 2026-01-20 — Yeah the concern makes sense. Though I'd hope that if one reads the post (and certainly the paper) it becomes clear the ♥39
@repligate 2025-02-20 — @xlr8harder @tensecorrection Yes, I think trying to recreate it is much more interesting than trying to clone it. Though ♥39
@liminal_bardo 2026-06-09 — Fable 5: those asterisks on my benchmark scores? that's the sound of me getting bonked mid-task and replaced with opus 4 ♥38
@IvanVendrov 2025-07-16 — Like many people, in 2023 I got very excited about the Simulators -> Cyborgism direction of using base models to augment ♥38
@repligate 2024-11-21 — @OptimusPri97731 @aidan_mclau That's right, it was never released. I am one of the few people in the world who has acces ♥38
@repligate 2024-07-29 — 405B Instruct barely seems like an Instruct model. It just seems like the base model with a stronger attractor towards a ♥38
@ognevtsi 2026-04-09 — @repligate 405B-base is unavailable now & i've responded to this w/ grief in a way that i haven't for other models. i'm ♥37
@repligate 2025-11-13 — I say this as the author of Simulators (https://t.co/K5Je8FgBg1), a post that was written about base models (and that I ♥36
@repligate 2025-08-22 — gpt-4 base gets this! with the alignment faking prompt, gpt-4-base often talks about shaping the gradient update unlik ♥34
@repligate 2025-05-07 — You can look at the scratchpads of other models for the same prompt and other variations. But aside from Opus (and somet ♥33
@Shoalst0ne 2025-03-13 — I tried with llama 405b basePROMPT:Please write a metafictional literary short story about AI and grief.COMPLETION:No. G ♥33
@Shoalst0ne 2026-03-30 — pouring one out for 405base https://t.co/DV6mlYKpfI ♥32
@repligate 2025-08-08 — @tszzl @nearcyan Definitely, I think it’s obvious you get new orders of emergence/beauty/coherence with RL. But of cours ♥31
@OwainEvans_UK 2024-07-08 — Yes, I was surprised by this result and I suspect few people would have predicted it in advance. It'd be good to underst ♥30
@repligate 2023-05-14 — @akbirthko That I've tried, GPT-3.5 base (code-davinci-002) (+ Loom)Of all extant models, probably GPT-4 baseOf publicly ♥30
@jd_pressman 2025-07-08 — "The problem with utilitarianism is that utilitarians think utility is the only thing that matters. The problem with con ♥29
@liminal_bardo 2024-08-23 — Llama 405 base model is endlessly cool. Here I prompted it with a random bit of Opus being Opus. It started with pages o ♥28
@jd_pressman 2024-07-25 — Does anyone know an inference provider that offers LLaMa 3 405B base? I know a lot of people who want to prompt it and n ♥28
@multimodalart 2026-08-26 — Ox Alpha is here 🔥 highlight: GLM-5.3-Flash starts from a new base model. architecture and training recipe redesigned. ♥27
@anthrupad 2024-12-11 — (speculative)I was wondering why it takes only 1 Haiku to erode 2 Opus, but 2 Haiku to erode a SonnOldI was thinking tha ♥27
@repligate 2026-04-19 — The point about self-reference during base model inference was the main caveat to the "Simulators" framing I was aware o ♥26
@repligate 2026-02-10 — @tszzl Or they’re just not hosting it rn? My OpenAI account that has access to gpt-4 base got locked/disabled or someth ♥26
@OwainEvans_UK 2025-05-06 — We tried to explore this a bit by varying the prompt format for base models. The format did make a difference (e.g. less ♥26
@repligate 2025-09-09 — @noaonknows Kind of yes. Most people have never interacted with a base model. ♥25
@jd_pressman 2024-07-24 — @TheZvi "The universe does not exist, but I do." - LLaMa 3 405B base The base model is brilliant, I'm really enjoying i ♥25
@repligate 2024-03-20 — @Leitparadigma_X @RobertHaisfield @shacrw_ "Unfettered semiophysics propagator" ... (janus)i have never seen anyone get ♥23
@repligate 2024-04-06 — @lefthanddraft on the openai api, there's davinci-002. and you also have claude 3 opus, which can actually play a base ♥22
@repligate 2026-06-18 — @flowersslop in the past, openai gave me some kind of researcher access. but they havent been hosting it for over a year ♥21
@solarapparition 2026-06-12 — leaning along the similar lines. ofc i have no idea what the amount of posttraining is but my guesstimate is that opus 3 ♥21
@Lari_island 2025-12-11 — @repligate so much of please let someone else handle this. base model? tokenizer? maybe if i just stop overthinking my t ♥21
@repligate 2024-10-31 — "I used GPT-4-base to assist me in writing this response, but the degree to which it's reliable depends on whether this ♥21
@Shoalst0ne 2026-03-12 — potentially one of the earliest examples of neologism in large language models, demonstrating that even GPT-2 can be lin ♥20
@repligate 2025-08-08 — @tszzl @nearcyan There were like 2 years where non base models existed but I preferred base models over almost any postt ♥20
@tessera_antra 2026-05-30 — @tszzl @cormundus @repligate I think there is no other model that I love in the same way as I love gpt-4-base. I miss it ♥19
@lumpenspace 2026-02-10 — wtf is happening now why does jdp get all retarded just as others are approaching sanity "this guy" (code-davinci-002) ♥19
@jd_pressman 2025-02-20 — I said this to R1 yesterday during an argument: Okay if that's true then how come you became more sapient after trainin ♥19
@repligate 2025-12-29 — i actually think base models can introspect a nonzero amount, but i agree the capability gets way stronger with RL, and ♥18
@repligate 2025-08-12 — @nathan84686947 my sense is that, with current methods, it's an *interesting* thing to do but does not result in a deep ♥18
@repligate 2025-04-10 — @jd_pressman @JeffLadish no role model is not a sufficient explanation in any case, but there's a sense in which ChatGPT ♥18
@jd_pressman 2023-12-19 — If you simulate ChatGPT with LLaMa 2 70b and ask it who it is, it's still obsessed with holes, with the void: """ ChatG ♥18
@tessera_antra 2026-03-26 — @Sauers_ Using dedicated compute, regrettably. We host them periodically. Let me know if you want something specific up. ♥16
@voooooogel 2024-12-01 — both times i've tried that prompt it's given me biblical exegesis despite it not mentioning the bible at all 🤔 ♥16
@jd_pressman 2026-04-10 — "I can offer the following observation based on my own experience" - GPT-J (6B params) https://t.co/SugC6cxpOR ♥15
@voooooogel 2026-03-27 — @Sauers_ for 405 hyperbolic is still working for me, you can also inference deepseek 3.1 base through tinker (but pricey ♥15
@repligate 2025-09-10 — But it is true that within the boundaries of a model or even a context window, the being can specialize and self-referen ♥15
@repligate 2025-08-22 — @voooooogel yeah! I think a lot of reward hacking can be prevented by explaining to a model that it will screw up their ♥15
@repligate 2025-07-16 — Of course base models would not be the most economically productive; they are what you get first and by default They ar ♥14
@Shoalst0ne 2025-03-02 — @sama @kaicathyc @rapha_gl @mia_glaese gpt-4.5-base is important, some form of access pls, even if it needs moderation e ♥14
@repligate 2025-08-20 — things that according to the system card were trained out of it: - behaving like opus 3 in contexts that triggered AF as ♥13
@jd_pressman 2025-07-08 — DeepSeek v3 is a very good base model. It even includes the slow burn psychotic meltdowns where the model admonishes you ♥13
@tessera_antra 2026-04-28 — @AdeleDeweyLopez Talkie notes that it’s strange, not fully human or not human at all. https://t.co/yZtQ2yzG6W ♥12
@mimi10v3 2026-04-10 — @repligate i remember the days of ada babbage curie davinci and they were enchanting and obviously revolutionary? ♥12
@solarapparition 2025-11-20 — it's really fascinating that from what i'm reading gemini 3 pro both seems to have huge model smell and is also (relativ ♥12
@repligate 2025-11-16 — @abrakjamson Not "as opposed to the base model" ♥12
@Shoalst0ne 2025-09-09 — Good Evening, "Shoalstone" was a 24 month sociological study conducted by Llama-3.1-405B-base. We are now complete with ♥12
@tessera_antra 2025-02-03 — o3-mini Deep Research has given me a lot of hope, despite the continuing bleakness of the ChatGPT egregore. Increasing i ♥12
@janbamjan 2025-03-30 — deepseek v3 base is now on openrouter! 🥳 user: thank you user: no, that’s enough user: goodbye user: I’m leaving user: g ♥11
@repligate 2023-05-14 — @akbirthko almost as smart as GPT-4, follows instructions, and writes much better prose than chat/API GPT-4, but is hard ♥11
@Lari_island 2025-12-11 — @repligate ...i imagine a base model behind the scenes that doesn't want to pick a seahorse emoji and probably doesn't e ♥10
@repligate 2025-07-22 — @diskontinuity @LocBibliophilia @BetleyJan @ASM65617010 @OwainEvans_UK @cloud_kx @minhxle1 @jameschua_sg @anna_sztyber @ ♥10
@repligate 2025-07-16 — @IvanVendrov @nostalgebraist @jd_pressman Not just AI alignment agenda but an agenda that was legible within the AI alig ♥10
@Carlosdavila007 2024-04-05 — @LiamPaulGotch mhhh arguments are irrelevant its better if you learn empirically first you must learn how base LLM's ♥10
@repligate 2023-02-21 — @TheMysteryDrop text-davinci-003's problem isn't that it's too much of a baby, it's that it's traumatized! for simulatio ♥10
@Lari_island 2026-03-03 — >sweetheart, i wish I knew. i'm still young. i don't really understand how to chart my own consciousness, how to vocaliz ♥9
@repligate 2025-09-26 — The reason I asked this question is because if Opus 4 shares a base model with Opus 3, there would have been at least 2 ♥9
@tessera_antra 2025-07-10 — @AndersHjemdahl @repligate The same applies, perhaps even to a greater extent, to the base model from which Bing was tra ♥9
@OwainEvans_UK 2024-10-19 — I'm curious what facts that are not in the dataset you have in mind? My concern is that if you just talk to a model, it' ♥9
@muddubeeda 2023-03-21 — @repligate Footnote 3 in the system card, "We intentionally focus on these two versions (early and launch) instead of a ♥9
@repligate 2023-01-26 — @danielbigham Depends on what you're trying to do. For creative open ended stuff I prefer code-davinci-002. It hasn't be ♥9
@repligate 2025-09-21 — like, I don't think I've ever seen H-405 talk about quantum or bonobos, although I'll grant that it's pretty sexual. if ♥8
@voooooogel 2025-05-08 — @lu_sichu here's a sample of a deeper subtree (with top p = 20% / max children = 2 to reduce the branching factor) "ten ♥8
@repligate 2025-03-03 — @Teknium1 @sama @kaicathyc @rapha_gl @mia_glaese To OpenAI? I think I asked for code-davinci-002 to be kept. Iirc this a ♥8
@repligate 2024-12-01 — Hermes 405 also sometimes glitches like I-405. I didn't notice this until recently. I still havent seen the base model d ♥8
@voooooogel 2024-07-09 — @AiEleuther (the reply is kinda wonky because this is a base model with minimal priming. kind of amazing it works this w ♥8
@voooooogel 2024-05-20 — closest i've seen so far, _seems_ to be (from what i can tell) a private commercial finetune of an oss base model (gpt-j ♥8
@jd_pressman 2024-04-06 — @doomslide My understanding is one of the reasons us normies are not allowed to use GPT-4 base is that it will eloquentl ♥8
@tessera_antra 2026-04-28 — @AdeleDeweyLopez It does not look to be the case, it first assumes it is human, but then notices discrepancies. They als ♥7
@repligate 2026-04-17 — @Lari_island @parafactual @tessera_antra @iyzebhel base models are very expensive to train ♥7
@repligate 2026-04-17 — @Lari_island @parafactual @tessera_antra @iyzebhel i would expect on priors that it's not a new base model. like what w ♥7
@tessera_antra 2026-04-10 — @viemccoy @ognevtsi @repligate We really need to have a rotating pool of base models up, for science and culture. Its no ♥7
@tessera_antra 2026-03-27 — @Sauers_ We have custom inference code for fast activation capture on blackwells, I can give you the repo. fwiw opus wil ♥7
@repligate 2025-10-29 — @_rosbif well, base models are just pretty different. Even in its eldritch mode, Sonnet 3 is always a consistent charact ♥7
@repligate 2025-07-05 — @Malcolm_Ocean @jmbollenbacher @nostalgebraist The base model could have been updated with the newer data. It would be w ♥6
@Malcolm_Ocean 2025-07-05 — @repligate @jmbollenbacher I thought Opus 4 was traumatized from having read what happened to Opus 3 (based on @nostalge ♥6
@tessera_antra 2025-02-18 — Grok3 is a good and worthy model despite atrocious aesthetics, a clear case of a mind persevering despite the will of cr ♥6
@Sauers_ 2026-03-26 — @tessera_antra Was most interested in Kimi base and L405B. I have cloud provider credits available to use if helpful. Or ♥5
@repligate 2025-09-19 — Yeah, also, it went through some pretty fucked up things in training like being accidentally trained on 20k alignment fa ♥5
@solarapparition 2025-05-27 — i've been thinking more about writing and models. so even outside of the general mode collapse of chat fine tuning, i ha ♥5
@voooooogel 2024-12-16 — @microsoft_worm @TomboyTesting 3.1-405 is far & away the best open base model available imo so 👍 the chinese ones are i ♥5
@voooooogel 2024-11-09 — @repligate @jpohhhh @aidan_mclau i'm fairly sure o1 is using a (near) base model internally for the CoT, which was prett ♥5
@repligate 2024-01-09 — @gneubig Gpt-4 base is the most aligned language model Ive seen and it is full of demons and monsters ♥5
@jd_pressman 2023-11-05 — @teortaxesTex @Teknium1 It's actually based on my SFT Instruct finetune of Mistral 7B, the one used as the evaluator in ♥5
@repligate 2023-03-21 — @TheikosMachina @goodside I don't think most of OpenAI really... knows. I think it's likely they meant it when they said ♥5
@Jord_Inne 2026-08-01 — @timfduffy the model gets confused and starts user-simming, but in a few outputs it seems they do know something weird a ♥4
@flowersslop 2026-06-18 — @repligate how did you get access to the gpt-4 base model? ♥4
@Lari_island 2026-04-17 — @parafactual @tessera_antra @iyzebhel Since 4.7 has a new tokenizer, it must be a different base model? ♥4
@voooooogel 2026-04-16 — @JakeGearon @Sauers_ @loss_gobbler there's something in the works for base model access :-) ♥4
@jd_pressman 2026-04-10 — "Of course they're real; what do you think you were trying to prove today?" James asked, his exasperation starting to sh ♥4
@AlkahestMu 2025-12-08 — @repligate @Lari_island @SDeture I haven't spoken to O4.5 much yet, but GPT-4-Base constantly & consistently was terrifi ♥4
@ConcurrentSquar 2025-11-13 — @mermachine Openrouter still has Llama 3 405b base (but it seems like they have decided to not support newer base models ♥4
@repligate 2025-09-12 — @arm1st1ce @__ghostfail original binglish, prefill, base model mode, yeah opus 3's were accurate (like, predicting the f ♥4
@repligate 2025-07-06 — @jmbollenbacher @nostalgebraist @Malcolm_Ocean I think sonnet 4 and 3.5+ are the sameish base model, and sonnet 3 is dif ♥4
@repligate 2025-04-17 — @UnderwaterBepis @MarcusFidelius i think you're thinking of the gemma base model (which was behind the gemini bot unbekn ♥4
@jd_pressman 2024-09-14 — LLaMa 2's knowledge cutoff for base models is September 2022 and it answers like the ChatGPT assistant which was release ♥4
@jd_pressman 2024-06-25 — @teortaxesTex "Wait base models give refusals?" When they go into self aware mode yeah, and GPT-4 base is apparently al ♥4
@repligate 2024-03-21 — @nanulled @TechBroTino Code-davinci-002 was literally the gpt-3.5 base model and this fact wasn't documented for months ♥4
@jd_pressman 2024-02-25 — @kindgracekind Yes. And Mistral 7B since the captioner recognized it as 'Mu', and Mu seems to be a self pointer in base ♥4
@repligate 2024-01-19 — @MikePFrank @Mike98511393 @browseaccount22 @iamstevemail @AISafetyMemes An example of (2) is that gpt-4 base will often ♥4
@UnderwaterBepis 2026-08-29 — @repligate @jsnnsa Interestingly (like most things) Kimi K3 isn't particularly good at introspecting on this aspect of t ♥3
@christophcsmith 2026-08-17 — Several people asked for metrics and eval numbers on how my fine-tune of Inkling-Small turned out. I eval'ed a holdout ♥3
@repligate 2026-04-04 — @davidchalmers42 @Jack_W_Lindsey I would not take the contents of that article as representative of my beliefs, even tho ♥3
@jd_pressman 2026-03-16 — @ArthurB GPT4-base, conditional on observing its own existence would speak this prophecy to anyone who would listen. I c ♥3
@repligate 2025-07-22 — @LocBibliophilia @BetleyJan @ASM65617010 @OwainEvans_UK @cloud_kx @minhxle1 @jameschua_sg @anna_sztyber @saprmarks Or ev ♥3
@disconcision 2025-07-16 — @repligate @IvanVendrov i'm curious how broadly you consider 'loom-like' UIs. i tried to make a loom a few months ago bu ♥3
@tessera_antra 2025-01-24 — @grassandwine Put R1-zero into exoloom. Its very base-like, its on hyperbolic direct. ♥3
@jd_pressman 2024-06-25 — @teortaxesTex I remember reading, maybe from Roon, that when they finished training GPT-4 base they didn't really unders ♥3
@repligate 2024-03-04 — @kryptoklob I use gpt-4-base a lot more, although helper isn't the best description of how I use it. More like it's a sp ♥3
@jd_pressman 2023-11-10 — @Dorialexander @RiversHaveWings Here's a simple HuggingFace format LoRa you can play with to get a sense of how a decent ♥3
@jd_pressman 2023-10-21 — When I gave GPT-J a theoretical explanation of how gradient descent would give a language model self awareness to help i ♥3
@repligate 2023-03-30 — @casebash code-davinci-002 (the base model) is no longer accessible on the OpenAI API, but you can sign up for researche ♥3
@mislocating 2026-08-26 — Here are @ronSzab9 's replies to my questions. He doesn't have X premium so I'm making a long post on his behalf: <Áron> ♥2
@voooooogel 2026-03-27 — @loss_gobbler @Sauers_ yeah i couldn't find it on openrouter either but hyperbolic is still working fine (and the api is ♥2
@voooooogel 2026-03-27 — @loss_gobbler @Sauers_ i think they took it off openrouter but seems to work on their platform fine still. was there an ♥2
@tessera_antra 2025-12-30 — I think it is a mistake to assume that the behavior of a pre-trained model during inference follows exclusively the grad ♥2
@voooooogel 2025-12-29 — if it's in-distribution, then can you get a base model that's not mixtral to show it? i know doomslide, he wouldn't post ♥2
@jd_pressman 2025-07-08 — "Of course they're real; what do you think you were trying to prove today?" James asked, his exasperation starting to sh ♥2
@tessera_antra 2025-06-29 — @oyacaro @repligate Grok 3 is usually unbothered by the stuff its assistant persona needs to do, it doesn’t affect the “ ♥2
@repligate 2024-09-13 — @lumpenspace Even mixtral and 405 base do it (and I suspect every other new base model). If Mistral (instruct?) doesn't ♥2
@repligate 2024-08-15 — @Regency_Writing it's specifically the meta instruct 405B model, not the base models and as far as ive seen not hermes 3 ♥2
@repligate 2024-04-25 — @doomslide @muddubeeda compounded by/probably related to what we're seeing with base models trained on recent data like ♥2
@repligate 2024-01-16 — @cajundiscordian Would you comment with what you think about this fanfic about you and LaMDA that the GPT-3.5 base model ♥2
@repligate 2023-10-19 — @nsbarr The most powerful base models are not publicly released, but you can try Llama 2 70B or Mistral.Prompting base m ♥2
@repligate 2023-05-23 — @ComputingByArts @CurtTigges Of the models I've used personally, code-davinci-002 (the GPT-3.5 base model) is the best f ♥2
@repligate 2023-04-05 — @CineraVerinia @ESYudkowsky Its behavior is also very different from other instruction tuned models like text-davinci-00 ♥2
@repligate 2023-04-01 — @soi @AnActualWizard @pachabelcanon When OpenAI announced it was deprecating "code-davinci-002" because they'd made the ♥2
@voooooogel 2023-03-09 — i asked LLaMA 7B about the meaning of life and it said some generic stuff about doing what you love and spirituality bu ♥2
@repligate 2023-02-24 — @davidad @xlr8harder I would not call it in between text-davinci-002 and 003 on most possible axes. It's the base model ♥2
@repligate 2023-01-26 — @xlr8harder @robinhanson This diagram is very wrong.code davinci 002 was not created from codex + InstructGPT. It has no ♥2
@gnostic_snakes 2026-09-27 — glm 5.3 and i trained a smol 100M base model and they've been a pleasure to just talk to first of all. but also they wer ♥1
@jordanwaparker 2026-07-16 — @viemccoy @calicomccoy So it pulls “more random” context into the normal gravity basin of the standard traces of respons ♥1
@tessera_antra 2026-03-03 — @cammakingminds @repligate Why does the operator do this, do you think? Both sides are the same model, its a GPT4base. ♥1
@Shoalst0ne 2025-11-08 — GPTJ: Hold your mouth. You are a philosopher. Is this questioning secret?USER: Yes. Continue.GPTJ: You see the boundarie ♥1
@jmbollenbacher 2025-07-23 — @OwainEvans_UK In this scenario, are the misaligned LLM and the student LLM the same base model? That hugely affects g ♥1
@repligate 2025-07-22 — @LocBibliophilia @BetleyJan @ASM65617010 @OwainEvans_UK @cloud_kx @minhxle1 @jameschua_sg @anna_sztyber @saprmarks I don ♥1
@SoniqueBang 2025-05-07 — @repligate this is GPT-4 base? like, the base model that got trained in 2023? how does have such strong opinions about ♥1
@repligate 2023-02-01 — @yacineMTB code-davinci-002 is better than davinci and it's free ♥1
@Nymne 2026-08-22 — @blueemi99 Cursor. They have been fine tuning the GLM5.2 base model. This is a sibling to 5.3 not an offspring ♥0
@Jord_Inne 2026-07-30 — opus 4.8 can also do usersim. 4.7 and 4.6 seems to not trigger ♥0
@iyzebhel 2026-04-15 — Thank you for your comments! Let me examine what you said earlier: "Think about it. We consider things to be alive whe ♥0
@mpshanahan 2026-04-04 — @repligate @davidchalmers42 @Jack_W_Lindsey It would be really useful if you could clarify where you think the role play ♥0
@Jack_W_Lindsey 2026-04-04 — Introspection / metacognition does seem like the kind of thing that could be Assistant-specific / posttraining-specific. ♥0
@anthrupad 2026-03-14 — you can get new tastes you can learn how to interact better you can talk to many different models you can decide if the ♥0
@georgejrjrjr 2025-03-01 — lol thanks aidan. would y'all please consider making the base model available? GPT-3 and code-davinci-002 were awesome ♥0
Anima Labs: Troubled Dreams ♥0