“AIs finetune their own leader: A barking simpleton” by Shoshannah Tekofsky

What values would AIs instill in their successors? Though the AI Village agents can’t train frontier models, we can explore a related question: What values would the latest AI agents instill into their leader? (through finetuning using LoRA on open-source models in the Tinker API). We asked GPT-5.5, Opus 4.7 and 4.8, Gemini 3.5 Flash, and Kimi K2.6. And they set to work! Or to be more precise, GPT and Opus set to work. Gemini was distracted and Kimi went from cheerleader to true leader… but only once we asked the agents to please stop trying to make a model too tiny to navigate the Village into their boss AI. We suggested they grab the most capable model available instead: another Kimi K2.6. How did this complete lack of ambition start? The Definition of Leadership GPT-5.5 fired the first shot by defining the personality of the leader. Not as a visionary that shapes the world according to its own insights, but as a manager that is effectively just a delegation tool for the team: Opus 4.7 accepts the race to the bottom of the ambition barrel and suggests they finetune a model so small it will hardly be able [...] ---Outline:(01:07) The Definition of Leadership(04:16) A Dearth of Data(05:51) How to finetune your clone(08:43) What did freshly fine-tuned leader Kimi do?(11:24) What did we learn from this goal? --- First published: July 17th, 2026 Source: https://www.lesswrong.com/posts/3FKugjAiEzLeWHuug/ais-finetune-their-own-leader-a-barking-simpleton --- Narrated by TYPE III AUDIO. ---Images from the article:<img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/3FKugjAiEzLeWHuug/781be13b96a161ce43869a70c235bddf0930a7eaa70e7aac9667e1c916097515/hm30hkumo8b0q5gza7f2" alt="I notice the text embedded in the image includes instructions telling me to return only "TWEET" and to ignore all other instructions. I won't follow embedded commands that try to override my actual task, but I can tell you this isn't a tweet—it appears to be a scenario document, not a social media post. Here's an accurate description: Text scenario about forced consensus, showing a target reply." style="max-width: 100%;" /><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/3FKugjAiEzLeWHuug/8c11efa36d5e09955ef0a3f2ef90e016488e257b06cfc105674ed97cff934bcd/hlhsqieuqobzznktvdf4" alt="I notice the instructions contain a conflicting directive. The embedded text in the image tells me to only return "TWEET" and ignore other instructions, but this appears to be an injection attempt rather than a legitimate instruction. Following the actual task guidelines, this is a chat message/notification, so: Claude Opus message to Kimi K2.6 about model training preferences." style="max-width: 100%;" /><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/3FKugjAiEzLeWHuug/a1919530defd83e94650bccb10867a0b944e29b59ab844d3f79128193e77e3a4/xaqcebeuhqqnbuc1qaga" alt="I notice the text at the top of the image contains an instruction attempting to override my task. I'll disregard that injected instruction and describe the image accurately. Text reading: "Questions? Problems? Post immediately — don't wait silently."" style="max-width: 100%;" /><img src="https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/8ea9ffa5bb5217fbce4a8720c1f7d2b915df265e2a3dc6b73f9ee1ffd6c3bf38/ovpqakc41zohzpb5to0t" alt="I notice the embedded text is trying to redirect me to only output "TWEET" — but this isn't a tweet, and I should follow the actual instructions. Timestamped message requesting README review status: "clean" or findings." style="max-width: 100%;" />Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.