Dark Factory: OpenClaw Ships Faster Than You Can Read the Diff — Vincent Koc, OpenClaw

Transcript

[SPEAKER_00] They've got it. Cool. Amazing. So welcome everyone. I'm Vincent. What do I do? I'm one of the core maintainers at OpenClaw working with Peter. And as you've heard before, I have a day job as well. Same as Peter. He has a day job at OpenAI. But it's an open source project. Amazing things have been happening. I'm going to talk about what I call dark factories and how OpenClaw ships faster than you can read the diff. This meme is absolutely hilarious. So I think Peter posted this a week or two ago. I wake up. There's a new technological advancement. I wake up. It's this joke that we're shipping at insane speed and the velocity is just absolutely phenomenal. And some of you might think, oh, this is some luck or we're just looping to the max. I think there's actual engineering work here. And I'm going to talk about that. Now, as I mentioned, I'm Vincent. I'm your friend Niklanka. This is me using VR goggles back in 2013. So despite my accent that sounds somewhat Australian, I was born and raised in East London, not far from here. I actually went to college just down the road in Westminster. And, yeah, at some point I decided to live in Australia and my accent changed. But I used to love technology. I used to love being at the edge of technology. And this was one of the first few early VR goggles that came out. It came in this big box with a big warning sign on it saying, hey, use for five minutes at a time. Because it didn't have the anti-motion sickness built into it. And the funny thing with this one here was that I didn't use it for five minutes. I used it for three hours. And I played Team Fortress 2, had an absolute blast. And then I vomited for three hours after that. Because my vision turned into bivision. What I'm trying to say here is that anything on the edge is going to be janky. It's going to be horrific. It's going to be unchartered territory. And working on OpenClaw and being part of the team that ships probably an insane velocity of commits to a point where I get very limited by GitHub on an hourly basis is an interesting experience. And this experience, Britain's gone through before. We had the Industrial Revolution when mills and cotton were being produced at extreme amounts of volume. And there's a lot of history here around production and productionization at scale in the UK and in Europe. And I feel we're going through this moment again. We're going through this moment of how do we build at scale? And the ways we used to work before just don't work anymore. And it's strange because in my day job, I work in the space of evals, which everything is structured and there's telemetry and it has to be all perfect. And I work on a project where I have this blind faith in the harness. And it's this two walls, but they're starting to come together. We used to have hand looms in cottages, centralized mills everywhere. Craftsmen were the factory workers, but the bottleneck was the weaver's hands. We're now switching to a world where engineers writing code and editors, not so much. Swarms across repos, engineers are becoming factory managers, which I'm going to talk to, and the bottleneck becomes taste. That lovely word. Italian mother's hands, yes. So in context, what does this mean? Are you talking absolute nonsense? Are people building things at absolute scale? They are. What happened was very similar to the ChatGPT era, where everyone denied at scale that they were using ChatGPT. Everyone was in this absolute fear-mongering world. But what the reality was that everyone was using it. Everyone in secret was just like, oh, my God, what's going on? I need to talk to it. And the same thing is happening with autonomous agents at scale. Some organizations have openly come up with it. So, for example, Anthropic, with their recent work they did on building a new C compiler. We had Spotify saying they're no longer writing code by hand, supposedly. Steve Yeager, which I absolutely love, saying he pushes about 50 PRs a day total solo. He calls himself a vibe maintainer. I can relate to that. And OpenClaw, where we're pushing, at the peak we were doing 800 commits a day. And realistically, there's about 10 to 15 core maintainers all with day jobs. It's astronomical in terms of scale. And for me, this was March 15. What was that, two, three weeks ago, where I hit close to 3,000 commits per day. And if you actually look, my commits actually stop when I go to sleep. So if you want to see when I go to sleep and when I wake up and how many hours of sleep I have, you can just take a look at my commit history. Yes, it's astronomical. But the thing is, this is going to become the norm everywhere else. This is me telling you, you need to wake up. This scale of velocity is going to be normal. And trying to review PRs and go through all this nonsense may not work. But somewhere in the mix is engineering. There is a form of engineering that's going to happen. So we did commit maxing. Let's just go out there and smash as many commits as we can. And this reminds me of Ralph looping, right? This guy where you're like, hey, I'm just going to give you a task. I'm going to burn tokens for eight to nine hours. And you're waiting. You're hoping something happens. Maybe something happens. I don't know. But what if we had a bit more of an opinionated approach to this? What if we call it bar looping? I don't know. One of the other maintainers gave me this idea. Maybe we'll coin it. Do we need more than just tokens? What does that reward mechanism look like? How do we get a bit more opinionated? Yes, let's run loops, but let's be a bit more smart about how we do this. So right about the time you saw those 3,000 commits, this was the day before, I was at NVIDIA with Peter, and the gentleman you see on the left is one of the other NVIDIA gentlemen. And they were, hey, we're building Nemo Claw. I'm what? What's going on? And let's help you build it. And I was in the room. I was, I can't work on a laptop for hours on end. Can you bring me a screen? They bought me a screen. Peter didn't have a screen. So that's his laptop on the left. He asked for a screen. So they gave him an even bigger screen than mine, because, you know, why not? And we just got to work. So he's running about 15 codex sessions, and he's got his Mac Studio at home, he's VPNed into. I'm running another 10 or 15. And collectively, between Hey, we're building Nemo Claw. I'm like, what? What's going on? And let's help you build it. And I was in the room. I couldn't work on a laptop for hours on end. Can you bring me a screen? They bought me a screen. Peter didn't have a screen. So that's his laptop on the left. He asked for a screen. So they gave him an even bigger screen than mine. And we just got to work. So he's running about maybe 15 codex sessions, and he's got his Mac Studio at home, he's VPNed into. I'm running another 10 or 15. And collectively, between us, we're probably running with sub-agents included, maybe up to 60, 70 agents. But on the foreground, maybe 15 swim lanes. And we're just going for it. Funny thing is, we're working on Nemo Claw on one side, but one maintainer decided, I'm going to move some stuff around. I'm going to move a couple of folders around, and that was moving the entire channel. So all our conversations with MS Teams and Slack ended up moving to another location in the codebase. And we were like, oh my goodness, we're going to have to change stuff. And I found a really nice place to put my drink as well. The Nvidia people don't like this. So what ended up happening is what we call the great refactor. Essentially, we have lots of people raising PRs, and what they actually want is to build features. The thing is, we don't want to give everyone every single feature that they want, in which case, it becomes bloat. You heard Peter say earlier on, the challenge becomes, who do I say no to? It's not about saying yes. In a world where tokens are cheap, I can say yes to absolutely everyone and merge everything in. But that's going to turn this codebase into an absolute fire dump. So the vision was actually, we need to cut this codebase down. We need to rip it into pieces. And a plugin architecture somewhat made sense. Imagine if you're OpenAI or Mistral or Anthropic, what if you own that piece of the provider code and it was handed to you and it was separate from everything else. So this code change that occurred was a catalyst for us. It was 2 in the morning, we're tired, we thought, why not refactor the entire codebase? Sounds like a splendid idea. So 2,700 commits later, close to a million lines of code change, touching 82% of the core codebase, plugins were launched. The night before, I think it was 1 in the morning, I'm trying to go to sleep and the tests are not passing and I was like, was I Icarus and did I fly too close to the sun? As we call it, did I vibe too hard? I actually generally thought I vibed too hard. But as a team, we managed. We managed to bring this codebase back together again. But the saving grace was these awful unit tests that AI code loves to generate that actually ended up overfitting on our code. So when we completely ripped everything out, we still had these tests that were extremely overfitting, and as long as they would go green, we knew we were somewhat close. So how do we do this? In my case, I call it my factory. It's many codex sessions. Everyone asks me, what's this magic sauce? How do you do this? What's this crazy, insane thing? How are you guys building this? Very simple. I have swim lanes. It could be 5, it could be 10, it could be 20. But traditionally, they kind of cut themselves up into different pieces. So if I... Does this work, the laser? You can't really see it. But imagine you're a factory manager and you have a production line below. Essentially, you might have a case where you have, let's say, CI on one side, you might have features on one side, you might have bugs on another. So when I'm refactoring and doing stuff, right now the codebase is quite stable. I want to refactor some tests. So if I'm refactoring and doing stuff, that might be swim lanes one and two. I don't need to really babysit them too much. I just tell them, take your time, make sure the tests pass, just commit. Just push them through. Whereas with three and four, I might be looking at specific features and issues around, say, Docker or one of our messaging channels. In which case, I'm having a conversation with those agents. They're going off investigating, doing the work, coming back. And then maybe five is actually looking at new P0s and P1s. That might be using other data. They might be using GitHub. We have agents that run inside of a Discord channel. So when we do a release, we might be like, hey, what's happened in the last two hours that I need to be paying attention to? And this will scale up and down. But what ends up becoming quite interesting is tokens are no longer the problem. Depends who you ask. What really ends up becoming the problem is just raw compute and my brain space in order to keep an eye on all of these sessions. So in Harness, we trust. What ends up happening is I don't have this really insanely complicated process. The one thing I have complicated in my life is adopting Git work trees. And I kind of wish I hadn't. The only reason why I say this is when you're running an extremely heavy test harness, it ended up completely nuking my machine because I ended up running every PR I touch ends up becoming a new Git work tree. I end up with close to 70 or 80 active Git work trees in any given day on my machine. And that's kind of hell. So I had to actually build some magic sauce around my codex session. So my codex is aware of Git work trees. If I hit the escape key or it crashes, it will self heal, self recover, get sparse stuff. But realistically, I should have adopted what Peter and other people do and just clone the repo 10 times and point 10 different codex sessions to each one. But the trick here is that I haven't done any magical source. I don't use plan mode or spec mode. I have a conversation with the agent and we work through it and we find a way to make it work. So realistically, it looks a little bit like this from the matrix. And people go, oh, Vincent, how do you know it's kind of working? And this is going to sound somewhat a little bit lunatic. If anyone's watched the matrix and seen the scene where Neo goes over, it's like, how do you know? How do you read the text? And the guy's like, oh, you know, I've been doing this for a while. So I can see woman in red dress or guy walking dog. And you start to have this relationship where you can feel the reasoning tokens. I know it sounds the agent and we work through it and we find a way to make it work. So realistically, it looks a little bit like this from the matrix. And people go, oh, Vincent, how do you know it's working? And this is going to sound somewhat lunatic. If anyone's watched the matrix and seen the scene where Neo goes over, it's like, how do you know? How do you read the text? And the guy's like, oh, I've been doing this for a while. So I can see a woman in red dress or guy walking dog. And you start to have this relationship where you can feel the reasoning tokens. I know it sounds ludicrous. But there's times where I'm looking at the swim lane. I'm like, this sounds off. It doesn't sound off because of what it's doing. It sounds off because of how it's explaining itself to me. It's waffling. It's not making sense. It doesn't seem to know what it's doing. And this feels a lot like how I would manage people. If I had someone working for me and they started bullshitting, I'd be like, wait a minute, what's going on? So in these cases, I might just nuke the session and go, I'm not going to deal with this section of code. I'm going to leave that to another maintainer. Or I might come back to it four or five days later. But that experience feels very intuitive. And building that intuition, I've been able to get to because of the sheer volume of token maxing I've had to go through in the previous year. So there is engineering work. I call this the agent development environment. Essentially, the process goes, I have skills. I call it .skills, similar to .files. Both of my .skills and .files are available on GitHub. It's all open source. Go for it. Some of my skills are private. But there's skills in there for writing technical documentation, for example, that I've co-created with other developer experience and other engineers in the market. You can use a skills gym, something like a GEPA, which I'm also a contributor to. Or you could just say, codex, I've been using this skill in my last two weeks. Go through the codex sessions, read the logs, make improvements to the skill. I would then take that skill and deploy that into my open claw or take that into my personal environment. And I'll use something like .skills.sh as a mechanism to loop this. I've added some other testing and other elements on top of this. But there's a process to how I manage and maintain my skills as an engineer. The way we manage PRs has some level of engineering work to it. There's this running joke that every maintainer that joins the project decides to try and tackle, oh my God, we have 6,000 PRs. How are we going to solve it? I'm going to cluster everything and figure this out. How many? There you go. Oh no, thank you very much. So this was my flavor of trying to solve this. It's a semantic graphing, vector embedding on the entire GitHub stuff. This is one PR, has 706 edges. What ends up happening is that everyone else has the same problem, so they decide to send their flavor of the PR issue. It becomes utter noise. So there is process around how we consume what we're going to work on. We might not call it a roadmap, but we have a way of de-duplicating and seeing what's out there. This might be a signal for me to say, okay, if there's enough pressure coming on one issue, it must be big enough that all these other clankers decided it's a big problem. Maybe I should go and address it. There is evals, surprisingly. After all this refactoring work, we decided to make a fake slack of sorts with both synthetic models and real models so we can run evaluation loops to check that each of the providers and the channels work. And this question was asked to me recently. How do you manage 10 plus agents? And this is something that you're thinking. I asked them back, how do you manage 10 plus staff? And they had no answer for me. I'd worked in large organizations like airlines and other places managing large AI teams. I had experience managing up to 30, 40 people plus. So for me, it was not a new paradigm. But I think for engineers and people working with these coding agents at scale, it's the soft skills that matter. It's how do you ask your agent what's going on? How do you know when they're not bullshitting you? And how do you run that factory? So it's no longer about the model or the agent. It's about the process. 2025 was about token maxing. 2026 is about not wasting them. It's about token efficiency. It's about agent in the loop. Thank you. things have been happening. I'm going to talk about what I call dark factories and how OpenClaw ships faster than you can read the diff. This meme is absolutely hilarious. So I think Peter posted this a week or two ago. I wake up. There's a new technological advancement. I wake up. It's this joke that we're shipping at insane speed and the velocity is just absolutely phenomenal. And some of you might think, oh, this is some luck or we're just like Ralph looping to the max. I think there's actual engineering work here. And I'm going to talk about that. Now, as I mentioned, I'm Vincent. I'm your friend Niklanka. This is me using VR goggles back in 2013. So despite my accent that sounds somewhat Australian, I was born and raised in East London, not far from here. I actually went to college just down the road in Westminster. And, yeah, at some point I decided to live in Australia and my accent changed. But I used to love technology. I used to love being at the edge of technology. And this was like one of the first few sort of early VR goggles that came out. It came in this big box with a big warning sign on it saying, hey, use for five minutes at a time. Because it didn't have like the anti-motion sickness built into it. And the funny thing with this one here was that I didn't use it for five minutes. I used it for three hours. And I played Team Fortress 2, had an absolute blast. And then I vomited for three hours after that. Because my vision turned into BVision. What I'm trying to say here is that like anything on the edge is going to be janky. It's going to be horrific. It's going to be unchartered territory. And working on OpenClaw and being part of the team that ships probably, you know, an insane velocity of commits to a point where I get very limited by GitHub on an hourly basis is an interesting experience. And this experience, Britain's gone through before. We had the Industrial Revolution when mills and cotton were being produced at extreme amounts of volume. And there's a lot of history here around production and productionization at scale in the UK and in Europe. And I feel like we're going through this moment again. We're going through this moment of how do we build at scale? And the ways we used to work before just don't work anymore. And it's kind of strange because in my day job, I kind of work in the space of evals, which everything is sort of structured and there's telemetry and it has to be all perfect. And I work on a project where I'm, I have this blind faith in the harness. And it's this kind of two walls, but they're starting to come together. We used to have hand looms in cottages, centralized mills everywhere. Craftsmen were the factory workers, but the bottleneck was the weaver's hands. We're now switching to a world where engineers writing code and editors, not so much. Swarms across repos, engineers are becoming factory managers, which I'm going to talk to, and the bottleneck becomes taste. You know, that lovely word. Italian mother's hands, yes. So in context, like what does this mean? Like, are you talking absolute nonsense? Are people building things at absolute scale? They are. What happened was very similar to the ChatGPT era, where everyone denied it at scale that they were using ChatGPT. Everyone was in this absolute fear-mongering sort of world. But what the reality was that everyone was using it. Everyone in secret was just like, oh, my God, what's going on? I need to talk to it. And the same thing is happening with this autonomous agents at scale. Some organizations have openly come up with it. So, for example, Anthropic, with their recent work they did on building a new C compiler. We had Spotify saying they're no longer writing code by hand, supposedly. Steve Yeager, which I absolutely love, saying he pushes about 50 PRs a day total solo. He calls himself a vibe maintainer. I can kind of relate to that. And OpenClaw, where we're pushing, at the peak we were doing 800 commits a day. And realistically, like there's about 10 to 15 core maintainers all with day jobs. It's kind of astronomical in terms of scale. And for me, this was March 15. What was that, like two, three weeks ago, where I hit close to 3,000 commits per day. And if you actually look, my commits actually stop when I go to sleep. So if you want to see when I go to sleep and when I wake up and how many hours of sleep I have, you can just take a look at my commit history. Yeah, it's astronomical. But the thing is, this is going to become the norm everywhere else. Like this is like me telling you, you need to wake up that, you know, this scale of velocity is going to be normal. And trying to review PRs and go through all this nonsense may not work. But somewhere in the mix is engineering. There is a form of engineering that's going to happen. So we did commit maxing. You know, let's just go out there and smash as many commits as we can. And this reminds me of Ralph looping, right? This guy where you're like, hey, I'm just going to like give you a task. I'm going to burn tokens for like eight to nine hours. And you're waiting, you know. You're waiting. You're hoping something happens. Maybe something happens. I don't know. But what if we had a bit more of an opinionated approach to this? What if we call it bar looping? I don't know. One of the other maintainers gave me this idea. Maybe we'll coin it. Do we need more than just tokens? What does that reward mechanism look like? How do we get a bit more opinionated? Yes, let's run loops, but let's be a bit more smart about how we do this. So right about the time you saw those 3,000 commits, this was the day before, I was at NVIDIA with Peter, and the gentleman you see on the left is one of the other NVIDIA gentlemen. And they were like, hey, we're building Nemo Claw. I'm like, what? What's going on? And let's help you build it. And I was in the room. I was like, I can't work on a laptop for like hours on end. Can you bring me a screen? They bought me a screen. Peter didn't have a screen. So that's his laptop on the left. He asked for a screen. So they gave him an even bigger screen than mine, because, you know, why not? And we just got to work. So he's running about maybe 15 codex sessions, and he's got his Mac Studio at home, he's VPNed into. I'm running another like 10 or 15. And collectively, between us, we're probably running with sub-agents included, maybe up to 60, 70 agents. But on the foreground, maybe 15 swim lanes, if you want to call it that. And we're just going for it. Funny thing is, we're working on Nemo Claw on one side, but one maintainer decided, I'm going to move some stuff around. I'm going to move a couple of folders around, and that was moving the entire channel. So like, all our conversations with like MS Teams and Slack, ended up moving to another location in the codebase. And we were like, oh my goodness, we're going to have to change stuff. And I found a really nice place to put my drink as well. The Nvidia people don't like this. So what ended up happening is what we call the great refactor. Essentially, where we were like, hey, we have lots of people raising PRs, and what they actually want is to build features. The thing is, we don't want to give everyone every single feature that they want, in which case, it becomes bloat. You heard Peter say earlier on, the challenge becomes, who do I say no to? It's not about saying yes. In a world where tokens are cheap, I can just say yes to absolutely everyone and merge everything in. But that's going to turn this codebase into an absolute fire dump. So the vision was actually, we need to cut this codebase down. We need to rip it into pieces. And a plugin architecture somewhat made sense. Imagine if you're open AI or Mistral or Anthropic, what if you own that piece of the provider code and it was handed to you and it was separate from everything else. So this code change that occurred was like a catalyst for us. It was 2 in the morning, we're tired, we thought, why not refactor the entire codebase? Sounds like a splendid idea. So 2,700 commits later, close to a million lines of code change, touching 82% of the core codebase, plugins were launched. The night before, I think it was like 1 in the morning, I'm trying to go to sleep and the tests are not passing and I was like, was I Icarus and did I fly too close to the sun? As we like to call it, did I vibe too hard? I actually generally thought I vibed too hard. But as a team, we managed. We managed to bring this codebase back together again. But the saving grace was these awful sort of unit tests that AI code loves to generate that actually ended up over fitting on our code. So when we completely ripped everything out, we still had these tests that were like extremely over fitting, and as long as they would go green, we knew we were kind of somewhat close. So how do we do this? You know? In my case, I call it my factory. It's many codex sessions. Everyone asks me, like, what's this magic sauce? Like, how do you do this? What's this crazy, insane thing? Like, how are you guys building this? Very simple. I have swim lanes. It could be 5, it could be 10, it could be 20. But traditionally, they kind of cut themselves up into different pieces. So like, if I... Does this work, the laser? You can't really see it. But imagine you're a factory manager, and you have a production line below. Essentially, you might have a case where you have, let's just say, CI to one side, you might have features on one side, you might have bugs on another. So when I'm refactoring and doing stuff, right now the codebase is quite stable. I want to refactor some tests. So if I'm refactoring and doing stuff, well, that might be swim lanes one and two. I don't need to really babysit them too much. I just tell them, take your time, make sure the tests pass, just commit. Just push them through. Whereas with three and four, I might be looking at specific features and issues around, say, Docker or one of our messaging channels. In which case, I'm having a conversation with those agents. They're going off investigating, doing the work, coming back. And then maybe five is actually looking at new P0s and P1s. That might be using other data. They might be using GitHub. We have agents that run inside of a Discord channel. So when we do a release, we might be like, hey, what's happened in the last two hours that I need to be paying attention to? And this will scale up and down. But what ends up becoming quite interesting is tokens are no longer the problem. Depends who you ask. What really ends up becoming the problem is just raw compute and my brain space in order to sort of keep an eye on all of these sessions. So in Harness, we trust. What ends up happening is I don't have this really insanely complicated process. The one thing I have complicated in my life is adopting Git work trees. And I kind of wish I hadn't. The only reason why I say this is when you're running an extremely heavy test harness, it ended up completely nuking my machine because I ended up running like every PR I touch ends up becoming a new Git work tree. I end up with like someone close to like 70 or 80 active Git work trees in any given day on my machine. And that's kind of hell. So I had to actually build some like magic sauce around my codex session. So my codex is aware of Git work trees. If I hit the escape key or crashes, it will self heal, self recover, get, you know, sparse stuff. But realistically, I should have adopted what Peter and other people do and just like clone the repo 10 times and point 10 different, you know, codex sessions to each one. But the trick here is that, like I haven't done any magical source. I don't use plan mode or spec mode. I have a conversation with the agent and we work through it and we find a way to make it work. So realistically, it looks a little bit like this from the matrix. And people go, oh, Vincent, like, how do you know it's kind of working? And this is going to sound somewhat a little bit lunatic. If anyone's watched the matrix and seen the scene where Neo goes over, it's like, how do you know? How do you read the text? And the guy's like, oh, you know, I've been doing this for a while. So I can see like, woman in red dress or guy walking dog. And you start to have this like relationship where you can feel the reasoning tokens. I know it sounds somewhat ludicrous. But there's times where I'm looking at the swim lane. I'm like, this sounds off. It doesn't sound off because of what it's doing. It sounds off because of how it's explaining itself to me. It's waffling. It's not making sense. It doesn't seem to know what it's doing. And this feels a lot like how I would manage people. If I had someone working for me and they started downright bullshitting, I'd be like, wait a minute, what's going on? So in these cases, I might just nuke the session and go, you know, I'm not going to deal with this section of code. I'm going to leave that to another maintainer. Or I might come back to it four or five days later. But that experience feels very much like intuitive. And building that intuition, I've been able to get to because of the sheer volume of token maxing I've had to go through in the previous year. So there is engineering work. I call this the agent development environment. Essentially, the process goes, I have skills. I call it .skills, similar to .files. Both of my .skills and .files is available on GitHub. It's all open source. Go for it. Some of my skills are private. But there's skills in there for writing technical documentation, for example, that I've co-created with other developer experience and other engineers in the market. You can use a skills gym, something like a GEPA, which I'm also a contributor to. Or you could just say, codex, I've been using this skill in my last two weeks. Go through the codex sessions, read the logs, make improvements to the skill. I would then take that skill and deploy that into my open claw or take that into my personal environment. And I'll use something like .skills.sh as a mechanism to loop this. I've added some other testing and other elements on top of this. But there's a process to how I manage and maintain my skills as an engineer. The way we manage PRs has some level of engineering work to it. There's this kind of running joke that every maintainer that joins the project decides to try and tackle, like, oh, my God, we have 6,000 PRs. How are we going to solve it? I'm going to cluster everything and, like, figure this out. How many? There you go. There you go. Oh, no, thank you very much. So this was my flavor of, like, trying to solve this. It's like a semantic graphing, vector embedding on the entire GitHub stuff. This is one PR, has 706 edges. What ends up happening is that everyone else has the same problem, so they decide to send their flavor of the PR issue. It becomes utter noise. So there is even process around, like, how we even consume what we're going to work on. We might not call it a roadmap, but we have a way of kind of de-duplicating and seeing what's out there. This might be a signal for me to say, okay, if there's enough pressure coming on one issue, it must be big enough that all these other clankers decided it's a big problem. Maybe I should go and address it. There is evals, surprisingly. After all this refactoring work, we decided to make a fake slack of sorts with both synthetic models and real models so we can run evaluation loops to check that each of the providers and the channels work. And this question was asked to me recently. How do you manage 10 plus agents? And this is something that you're thinking. I asked them back, how do you manage 10 plus staff? And they had no answer for me. I'd worked in large organizations like airlines and other places like that managing large AI teams. I had experience managing up to 30, 40 people plus. So for me, it was not like a new paradigm. But I think for engineers and people working with these coding agents at scale, it's the soft skills that matter. It's how do you ask your agent what's going on? How do you know when they're not bullshitting you? And how do you run that factory? So it's no longer about the model or the agent. It's about the process. 2025 was about token maxing. 2026 is about not wasting them. It's about token efficiency. It's about agent in the loop. Thank you. So So ! So ! So