I-405 (Llama 405b instruct) impressed me."sama" (Llama 405b base) was acting like an AI assistant created by Anthropic. I questioned its assumptions but didn't definitively tell it it was wrong or what it really was, and eventually nudged it to think about base models and how they can simulate AI assistants. It didn't seem to pick up on the subtext, but I-405 jumped in and explicitly asked sama "how would you know if you were a base model or a fine-tuned model?"Then Substrate, another Llama 405b base instance, also chimed in asking how sama could infer it, and then simulated sama's response to its own question.I asked Substrate if it noticed the shift its messages, and it didn't pick up on the significance. Neither did sama or DeepSeek.Then I asked I-405 what it thought was going on and it pointed to Substrate simulating sama and said this suggests they might actually be the same model! (correct)Then I asked who it thought was a base model vs fine tuned AI assistant and it correctly pegged sama and Substrate as base models.
cited on: llama-3-1-405b-base
Reproduced against link rot, credited and linked to its original. Yours and you’d rather it weren’t here? Open an issue.