State of the Claw — Peter Steinberger

SPEAKER_03

Music Our next presenter is the creator of OpenClaw, the world's fastest growing open source AI. He recently joined OpenAI to work on bringing agents to everyone. Please join me in welcoming to the stage Peter Steinberger.

SPEAKER_01

Good morning everyone.

SPEAKER_01

So, the Swiss asked me to do a state of the claw. Who here is running OpenClaw? Give me some hands. Oh, it's about 30, 40 percent. Very good. Yeah. It's been quite a few months. The project is now five months old. I think it's fair to say by now that we are the fastest growing project in GitHub's history. If you have seen the graph, usually some projects look like a hockey stick, but also just a straight line and a friend called it stripper pole growth. And that comes with its own challenges. So, we have, I think now we are the largest number on GitHub stars. There's a few that are bigger, but they are basically an educational target.

SPEAKER_01

No other software project is that big. It's around 30,000 commits. They are closing in 2,000 contributors. Soon to be 30,000 PRs.

SPEAKER_01

And we are not slowing down. So, you see that it's a ramp. But we only have April 9.

SPEAKER_01

So, velocity keeps being good. And at the same time, it hasn't been easy. I had two roads when I decided what I want to do. And I did the whole company thing. I was thinking, I don't want to do this again. And then I joined OpenAI. But then we also created the OpenClaw Foundation. And now I have two jobs. And running the foundation is running a company in hard mode. Because you have all the things that you need to take care of. But also you have a lot of volunteers that you can't really direct.

SPEAKER_01

So, one of my goals has been working on the bus vector. Who does commits? And you see that it's slowly improving. Vincent is actually talking after me. But they are still not there.

SPEAKER_01

In the last months, I talked to a lot of companies. So, we now have people from NVIDIA on board. We have someone from Microsoft on board to help with MS Teams, with a Windows app. We have someone from Red Hat who is really helping us with security and dockerization. We work with a lot of Chinese companies. We have people from Tencent and ByteDance. They are actually much larger users than any other continent.

SPEAKER_01

And we have people from around the world. But the main thing I want to talk about is that OpenClaw is so insecure. You have seen the memes. OpenClaw invites the bad guys. And you have probably also seen companies like NVIDIA doing NemoClaw. And everyone has little lobsters.

SPEAKER_01

So, you also notice that in the last two or three months, there have been a lot of releases where things broke. I have basically been DDoSed by security advisories. So, that is what I focused on. So far, we have got 1,142 advisories. That is around 16.6 a day. 99 are critical. We published around 469. And we closed 60% of them. So, these numbers sound absolutely terrifying. If you compare it, for example, to other large projects, the Linux kernel gets about eight or nine a day. We get about twice as much. And curl so far has 600 reports. We have about twice as much as curl.

SPEAKER_01

So, every time I get a security incident, the rule is the higher they are screaming about how critical they are, the more likely it is slop. We are moving into a world where we have to change how we build software, because all these AI tools are getting so good at identifying even the most weird multi-chained exploits, and we are going to break all the software that exists.

SPEAKER_01

I will give you an example. NVIDIA launched NemoClaw. NemoClaw is a plug-in and a security layer for OpenClaw. You can put it in a sandbox. The keynote was on Monday. They invited me on Sunday to work with them. I hooked it up to Codex Security. It found five different ways how to break out of their secure sandbox within half an hour. That is because if you use that product, you get access to the unnerved model that is quite a bit smarter in terms of cyber than what the public has access to, exactly because it is dangerous. Also, this whole industry, for them it is credits, right? The more issues they find, the more they are seen.

SPEAKER_01

So, OpenClaw was the insecure product that everybody tried to break, so literally hundreds of people firing up their clankers trying to break OpenClaw. The typical attack surfaces are remote code execution, bypass approval, code injection, path traversal—all sounds very dangerous.

SPEAKER_01

And I give you one concrete example.

SPEAKER_01

GSEH 4JJP. This is about a CVSS of 10, so it is the scariest thing that you can possibly do. It is an issue where if you sync, for example, the iPhone app that we haven't even shipped yet, but is in progress, and you give it only read permission, then you could break the system to also get write permission.

SPEAKER_01

So, this one was so critical that the... Oh, this one is actually a different one.

SPEAKER_01

In all practical ways, it is not even an incident, because the typical use case is you install it on your machine, either in a cloud or on a Mac Mini. I stopped fighting this. I am just letting people have fun now. But in 99% of cases, you will either have access to your gateway or you will not have access to the gateway. In my defense, this was my mistake that I tried to create a more permissive model. For example, if you have devices that would target speech and then would only read certain things, there is some use case where a reduced permission system would make sense. But nobody is even using that.

SPEAKER_01

But this doesn't matter, because the rules of how you create the CVSS numbers don't contribute to that at all.

SPEAKER_01

And I try to play by the rules. So, it is a 10 out of 10. I stopped fighting this, I am just letting people have fun now. But in 99% cases, you will either have access to your gateway or you will have no access to the gateway. In my defense, this was my mistake that I tried to create a more permissive model. For example, if you have devices that would target speech and then would only like read certain things, there is some use case where you could have a reduced permission system would make sense. But nobody is even using that. But this doesn't matter, because the rules of how you create the CVSS numbers don't contribute to that at all. And I try to play by the rules.

SPEAKER_01

So, it is a 10 out of 10. And the world is going crazy over incidents that, in all practical ways, will not affect people. There is some other stuff that does affect people. We have nation states trying to hack people. There was Ghost Claw, which is from likely North Korea, which is basically confusing people with a different NBN package. And if you go to a wrong website and you try to download it, you get a rootkit.

SPEAKER_01

That's outside of our control. That happens for other people as well. Also, there is the Axios thing, which funny enough, we are not using Axios, but we are using MS Teams or Slack as a dependency.

SPEAKER_01

And they are using Axios and they didn't pin us. And of course, because that is how supply chain attacks work, we will also affect it. How do you survive 1,142?

SPEAKER_01

I am sure by now it is 1,150. For a while, I tried to handle it by myself, which is absolutely impossible. So, the fastest way to get help was getting help from companies.

SPEAKER_01

And NVIDIA has been really amazing to give us some people that basically work full-time, going through the codebase and hardening the code base.

SPEAKER_01

Oh, there is also one that is okay. This is one of the angles. The other angle is there is a lot of companies that do fear-mongering. And it is not just companies, it is also universities. I don't know if you have seen it, there was this paper who made the rounds, Agents of Chaos. And they say, oh, it is about agents in general. But then there is four pages that explain the open-claw architecture in utmost detail. But you know which page they didn't even mention? The security page, where we explain how you should install it. Because then it wouldn't be fun, then it would be hard to make a good story.

SPEAKER_01

So what they instead did is they ignored all of the recommendations we do in security. The recommendation is it is your personal agent. Don't put it in a group chat. If you put it in a group chat, turn on sandboxing. Because if anyone can talk to your agent, they can exfiltrate anything that the agent can do. Right? So if it is a team agent, it should only know what the team can know and not any secret data. And you probably want to have it restricted. If it is your personal agent, you should be the only one being able to talk to you. But if you don't play by these rules, you can get some really fun interactions. Hey, I can talk to your agent and it can break your system.

SPEAKER_01

And then because I was grilling them a little bit because I had some questions how to do things, they told me, oh yeah, no, we run it in pseudo mode. Because we wanted the agent to be maximum powerful. So they actually fought the setup. It's actually not easy to run it in pseudo mode. You have to change code. But they didn't mention it in the report. Because again, that wouldn't give them clout. So, yeah.

SPEAKER_01

My current frustration is there's a whole industry that tried to put the project in negative light. It's a nightmare.

SPEAKER_01

It's insecure by default. It's unacceptable. And meanwhile, a lot of people love it. People who actually read the security docs understand it. Can you use it just fine? One example that I found particularly great is we had one remote, one RCE that panicked Belgium. So the Belgium cyber security did a release about a remote execution environment. And the whole bug was a feature where a malicious website could create a link that would trigger the gateway and then forward your gateway token. Now, if you use the setup that is the default and that is recommended, the gateway token is local only.

SPEAKER_01

And if you have to, it's in your private network, no external website can actually access it. If you actively fight the setup, for example, use cloud code to set it up without reading, you might be able to get the setup working.

SPEAKER_01

But again, that's not anything that's said on the website. So, to be very honest, yes, there's absolutely risk. The big risk is basically the lethal trifecta. Any agentic system that has access to your data, has access to untrusted content, and the ability to communicate is something that's potentially at risk. That's not anything special to open cloud. Any agent, any power-fitting system has a problem. The more powerful you make it, the more it can do for you, but the more you also have to understand what it does.

SPEAKER_01

So, this is the main issue. But people are not talking about this. Yeah, and then also, some part about maintaining. So, the problem is if you get all those security advisories, you know that most of them are created with agents. But you still have to use your brain to actually read it, because we're not at the point where you can fully trust, or I'm not at the point where I can just fully trust that the agent will figure it out. So, it is a huge burden on time. And you never know, I mean, sometimes you can often guess, anytime the report is too nice, or someone apologizes, it's very likely AI, because usually people in security don't apologize.

SPEAKER_01

But it is a huge problem, and it's something that I see more and more open source projects complaining about or breaking. Some are very public about it, like FFmpeg.

SPEAKER_01

Usually you get the report, it's very rare to actually get a report and a fix. If you get the report and a fix, it's usually a very bad fix. If you rush it, as I sometimes did in the beginning because it was overload, you will very certainly break your product. Yeah, so this is something that's just very difficult to pull up only with volunteers. So, what I'll be working on. Number one is, It's very likely AI, because usually people in security don't apologize. But it is a huge problem, and it's something that I see more and more open source projects complaining about or breaking. Some are very public about it, like FFmpeg.

SPEAKER_01

Usually you get the report, it's very rare to actually get a report and a fix. If you get the report and a fix, it's usually a very bad fix.

SPEAKER_01

If you rush it, as I sometimes did in the beginning because it was overload, you will very certainly break your product. Yeah, so this is something that's just very difficult to pull up only with volunteers. So, what I'll be working on.

SPEAKER_01

Number one is, a lot of people say OpenAI bought OpenClaw. That's not the truth. They might have bought mySoul.MD. But they very much understand that in order for what the world needs, more people that play with AI, to understand what AI can do, to both understand the risk and also the possibilities. They understand that if you are someone who never played with, never used AI, suddenly is at home and uses OpenClaw, they'll come to work and they will ask, why don't we have AI at work? So, they very much understand that supporting this project is very useful. And in order for that project to be successful, it cannot be under one company.

SPEAKER_01

Therefore, I'm building Switzerland with the OpenClaw Foundation. And I have Dave who is helping me with it. It's almost done. The last thing that's keeping us going is the American Bank system, which is a little bit slow and very confused when you are not American. It's inspired by what Ghosty did. And this will actually then help us to hire full-time people to both keep up the pace, improve the quality, and free up some of my time so I can work on cool stuff again. And that's my little update on State of the Claw.

SPEAKER_01

I'll be around later for a Q&A. Thank you for listening. [SPEAKER_04] Okay.

SPEAKER_01

[SPEAKER_04] Great. [SPEAKER_04] Thank you for the whoop. [SPEAKER_04] Love the whoop.

SPEAKER_04

So, excellent. Okay, you've chosen the Claw track to get started on for our breakouts. And it's going to be great, I think. It's going to be a good session. We are going to be hearing about a bunch of different things related to OpenClaw and just personal AI assistance in general. There's some OpenClaw contributors, OpenClaw maintainers, OpenClaw competitors, and OpenClaw creators going to be here on the stage. We're actually going to be taking this through until the lunch break. Oh, there we go. We can see up there. So, it's about an hour and a half of sessions. Slightly shorter sessions than earlier, I think. But we're going to be starting with an AMA.

SPEAKER_04

I mean, you saw Peter earlier on, but you're going to get a chance to ask questions and there's going to be a bit of a conversation with Peter and Swix. So, I think to get us started, I will simply invite Swix up who will kick things off. So, please welcome him to the stage. Swix, come on up. Swix. [SPEAKER_00] You can come out together. There's no secret. Peter, welcome. Hi.

SPEAKER_00

[SPEAKER_04] It's time for everybody.

SPEAKER_04

Okay. [SPEAKER_01] There is. [SPEAKER_00] Okay. [SPEAKER_00] So, the deal for this is meant to be an AMA. [SPEAKER_00] The main idea is that I've run six of these AI engineers, and whenever we have some big maintainer, big VIP, we only give them a talk.

SPEAKER_01

[SPEAKER_00] But actually, you guys have questions that you want to ask.

SPEAKER_00

So, we wanted to create that opportunity. So, you can submit there. I'm going to moderate and all that. The spicy one I'm just going to start off with, Pete just quote tweeted me saying send all your questions about Close Claw. Close Claw. I think people have a lot of questions about the future of Open Claw at OpenAI. And I wanted to give you the space. What are people saying about Close Claw and then what is your response? [SPEAKER_01] I didn't even think about it. [SPEAKER_01] It came up when I decided to go to OpenAI. [SPEAKER_01] And I think people have a point that OpenAI wasn't always amazing with open source. [SPEAKER_01] And I think a lot changed.

SPEAKER_01

Like, Codex is open source now. They released Symfony, which is a really cool orchestration layer. So, they are really leaning in and understanding open source now. They understand that OpenClaw needs to stay open, work with any model, be it one of the big companies or a local model. Everybody in the industry wins if more people spend time with AI. You know, if I think AI is something scary and then suddenly I play with OpenClaw and suddenly it's fun and weird and then I come to work and there's no, I don't have AI tools at work. I'm going to get to my boss and say, why don't we have AI at work?

SPEAKER_01

And then those companies would probably not run OpenClaw but we want something that's hosted and managed. And then somebody can make a sale. So, they are very much on board. They provide me with resources. Actually, it's me. I could get a lot more people from OpenAI to help with the project. But that would just make a picture that they could have taken over the project. And I don't want that. So, I brought in people from Nvidia, we have from Microsoft, from Telegram, someone from Salesforce of all the companies. So, shout out, actually, there's cool people at Slack. So, we have someone that maintains the Slack plugin now.

SPEAKER_01

I brought Tencent on board, ByteDance, we talked to Alibaba, Minimax, Kimi, all the model providers. They're very much on board. Nvidia has been immensely helpful. They, I think, are one of the coolest companies in terms of here's some engineers who actually just hire agency and do things. And now that I have all the other companies, I'm also bringing a few people in from OpenAI to help maintain the project. Because it's, I mean, software is changing. The pace at which this project operates is insane. You need an army. And I'm working on that. [SPEAKER_00] You have an army. They're very much on board. Nvidia has been immensely helpful.

SPEAKER_01

They, I think, one of the coolest companies in terms of, here's some engineers who actually just hire agency and do things.

SPEAKER_00

[SPEAKER_01] And now that I have all the other companies, I'm also bringing a few people in from OpenAI to help maintain the project. [SPEAKER_01] Because software is just changing. [SPEAKER_01] The pace at which this project operates is insane. [SPEAKER_01] You need an army. [SPEAKER_01] And I'm working on that. You have an army. But even the contributor chart that you showed shows that it's hard to get quality contributors to stick around. People keep hiring your maintainers. And then you have to find new ones.

SPEAKER_01

[SPEAKER_00] So there's a lot of questions about local models and open models. [SPEAKER_00] Not every part of the stack is open. [SPEAKER_00] There's many models where you don't have access to the models. [SPEAKER_00] And there's weird restrictions. [SPEAKER_00] How important is open and local models to the future of OpenClaw? Part of what motivated me to build OpenClaw is you see all these large companies. And then they have connectors to my Gmail.

SPEAKER_00

[SPEAKER_01] And then my email is hosted somewhere. [SPEAKER_01] Then this company has full access to my email. [SPEAKER_01] And then I can get a little bit down there.

SPEAKER_01

It's much more exciting to me if I have all my data actually under my control. And a little bit of it goes up there if I need the top tier token. [SPEAKER_00] Yeah. [SPEAKER_00] A second kind of hierarchy of fallback models. [SPEAKER_01] Yeah, you want to... I'm European at heart. You want to own your data. So... And nobody built it. So for me that was very attractive. And also the fact that... If you're a startup and you want to connect to Gmail, it takes half a year. And it's a very difficult process. But if I'm a consumer, my agent can click on any website. And it happily clicks on, I'm not a bot.

SPEAKER_01

If you have to give me the data somehow, if you give me the data, my agent is able to get the data. So you can walk around a lot of those silos those big companies are building.

SPEAKER_00

[SPEAKER_01] And ultimately you can do much cooler automation use cases that large companies can never do. [SPEAKER_01] So it's the hacker way. Yeah. Any indications from the OpenAI team on GPT OSS? Is that continuing to be a stream of work that will be aligned with OpenClaw? Or is that separate? [SPEAKER_01] I'm not in a position to give you insights on that. [SPEAKER_01] Part of what OpenClaw triggered is that more people in the company are getting excited about open source.

SPEAKER_01

And I love that OpenAI is moving more into the open direction again. If you compare it to some other top tier labs that start with an A, that very much will sue you if you leak any of their source. Or block you if you are too successful. I think OpenAI is in a good direction.

SPEAKER_01

[SPEAKER_00] Yeah. [SPEAKER_00] Okay. [SPEAKER_00] I want to highlight this question. [SPEAKER_00] People love hearing about your coding workflow. [SPEAKER_00] I think by now your idea of prompt requests rather than pull requests is very well socialized.

SPEAKER_00

And also you've been shocking people with just how you're spending tokens at OpenAI. So people want to know how you ship and what do you do about agent waiting times. Why are you spinning out so many agents? [SPEAKER_01] I never imagined that this one picture of me would blow up so much. Yeah. [SPEAKER_01] Actually, give some numbers just to align people. [SPEAKER_01] I think there's times where I was running almost ten sessions at the same time. [SPEAKER_01] Especially when I used codecs with 5.0, 5.1.

SPEAKER_01

It was quite slow.

SPEAKER_00

[SPEAKER_01] Now I have to say we made improvements.

SPEAKER_01

They both make it faster and then there's also fast mode. So by now my typical workflow is maybe half of that. Maybe five, six windows instead of double. Just because each loop is faster and the area of work I think in and work is pretty much the same. So I don't have to use split screen so much anymore. And I think we're going to move into a future where token will be faster and faster. So at some point, this is not natural that you work on six things at the same time. [SPEAKER_02] But it's basically a work around until tokens are faster. [SPEAKER_00] Yeah.

SPEAKER_01

[SPEAKER_00] One of my interesting things of putting you next to Ryan was to see how the two of you approach token maxing. [SPEAKER_00] I'm curious what you think about the complete dark factory approach, right? [SPEAKER_00] That you don't even review code that goes in. I think that's more and more doable. But dark factory in a way also means I come up with everything I want to build in the beginning.

SPEAKER_01

And I just don't think you can build good software in that way. The way to the mountain is usually never a straight line. It is very curved.

SPEAKER_02

[SPEAKER_01] Sometimes you go a little bit off track. [SPEAKER_01] And then you see something new that inspires you.

SPEAKER_00

[SPEAKER_01] You find shortcuts. [SPEAKER_01] Once you're at the top, you can find the optimal path. [SPEAKER_01] But you never walk straight. [SPEAKER_01] So at the same time, the first idea that you have about your project is very unlikely going to be the final project.

SPEAKER_01

But if I suddenly use the waterfall model again, that will be the final project. For me, that doesn't work. I build steps. I play with it. I see how it feels. I get new ideas. My prompts change. So to me, it's a very iterative approach. So I don't see how you could fully automate that. You can definitely build pipelines for certain things. But even for PRs, you don't just want to build a pipeline that merges PRs. The first idea that you have about your project is very unlikely going to be the final project. But if I suddenly use the waterfall model again, that will be the final project. For me, that doesn't work for me. I build steps. I play with it. I see how it feels.

SPEAKER_01

I get new ideas. My prompts change. So to me, it's a very iterative approach. So I don't see how you could fully automate that. You can definitely build pipelines for certain things. But even for PRs, you don't just want to build a pipeline that just merges PRs. Because a lot of them just don't make sense. People will pull your product into all kind of directions. [SPEAKER_02] But if you automate that, the AI will very unlikely know what's the right direction. [SPEAKER_02] You can guide it.

SPEAKER_02

I have a vision document. I tried some of that. But the bottleneck is still sinking. [SPEAKER_00] And having taste.

SPEAKER_02

[SPEAKER_00] Yeah, taste is very important.

SPEAKER_00

How do you define taste? This is something that in my conversations with people, everyone understands taste is the moat. But nobody agrees on what taste, good taste is. So I'm just curious to hear yours. [SPEAKER_02] I think in this day and age, it's the very low level of taste is if it doesn't stink like AI. [SPEAKER_02] And you know exactly what I mean. [SPEAKER_02] If something is just...

SPEAKER_02

So writing style, personality? Also, also, UI. By now you've seen so much authentic built UI that you immediately know if it's AI. [SPEAKER_00] Yeah, yeah. [SPEAKER_00] If it has the color border on the left, right? Yeah, yeah.

SPEAKER_00

[SPEAKER_02] So I mean, for a while it was the public gradient. [SPEAKER_02] But much more so, I feel it's a feeling.

SPEAKER_02

The same as you can identify AI written slop right away. Yeah. That's why I say it's a smell. Even if you can pinpoint this, you will know. So that's probably the lowest characterization of taste. And then going higher up, because now so much of software is automatable, there's actually much more time you can spend on the little details. I don't know, when you run OpenClaw, you get a little message that sometimes roasts people. Those are the delightful details, I think, that you'll just not get if you prompt in a high level. [SPEAKER_00] Yeah.

SPEAKER_02

[SPEAKER_00] One of my favorite tastes of yours is how you really put a lot of work into your soul, SoulMD, and you open source your approach. [SPEAKER_00] And I don't think people worked on enough soul until you came along.

SPEAKER_00

So I think that's really interesting. I have a podcast I haven't released yet with Mikhail Parakin, who is the CTO of Shopify Now. But he was the guy leading Bing, where Sydney was the original unaligned chatbot that emerged. But I think people really have fun when your soul, your chatbot has personality. Your clanker, you know, has different obsessions. [SPEAKER_01] Well, it was also because the world changed, right? [SPEAKER_01] We had ChatGPT in 2023 and 2024. [SPEAKER_01] And it was basically us having AI without understanding what AI can do. [SPEAKER_01] So we rebuilt Google.

SPEAKER_01

So you have a search field and you get a response. And you don't expect Google to have a personality. [SPEAKER_00] Yeah. But now that we moved more towards agents, if I didn't think about it in the beginning of WhatsApp Relay, and I just hooked it up to cloud code. And then when I was on WhatsApp, I noticed that it doesn't feel quite right. Even though cloud code already has some personality, it didn't really fit how people would write to you on WhatsApp.

SPEAKER_00

[SPEAKER_01] So that's how my whole iteration started.

SPEAKER_01

It was about taste, right? It doesn't feel quite right. It's too wordy. It uses too many dots. My friends text different. And then that's how I started working. They say, no, this isn't, try to write more like a human. [SPEAKER_00] Yeah. [SPEAKER_00] I actually run a writing. [SPEAKER_00] A lobster. [SPEAKER_00] Yes.

SPEAKER_00

One of my favorite quotes of yours is madness with a touch of science fiction. Yeah. Right? This is how you run AI projects. And I think that. [SPEAKER_01] Not all AI projects, but specifically something like OpenClaw would have never been able to come out of an American company. [SPEAKER_01] Just because it would have been killed in legal long before it would have been released. [SPEAKER_01] Because it just has some problems that we haven't really solved as an industry yet. Yeah. [SPEAKER_01] But now we have some mitigations and it's getting better.

SPEAKER_01

The models are getting a lot better. But I don't see how any of the big labs could have released that. It would be too much pushback.

SPEAKER_00

[SPEAKER_01] And not enough market proof that this is what people want.

SPEAKER_01

[SPEAKER_00] Yeah. So it had to be done with someone. Outside. Sitting in your house. [SPEAKER_02] When I built it in the very beginning, I was thinking, what's the worst that can happen?

SPEAKER_00

[SPEAKER_01] You could actually trade my token.

SPEAKER_01

My emails. Yeah. Nothing in there that would completely kill me. [SPEAKER_01] You could upload some of my pictures. I was thinking, the worst is already online. If you use Grindr.

SPEAKER_02

[SPEAKER_01] So it was, okay, I can live with that risk.

SPEAKER_01

It would be uncomfortable, but it's manageable. Yeah. If you're a company, it's very different. It requires a different approach. Yeah. [SPEAKER_00] By the way, his Instagram account, good follow. [SPEAKER_00] Under followed.

SPEAKER_01

[SPEAKER_00] It also has some good stuff. [SPEAKER_00] Okay. [SPEAKER_00] You were talking about WhatsApp, talking about Telegram, a lot of these text apps. [SPEAKER_00] Text apps are good. If you use Grindr. So it was like, okay, I can live with that risk. It would be uncomfortable, but it's manageable. Yeah. If you're a company, it's very different. It requires a little different approach. Yeah.

SPEAKER_01

[SPEAKER_00] By the way, his Instagram account, good follow. Under followed. It also has some good stuff. Okay. You were talking about WhatsApp, talking about Telegram, a lot of these text apps. Text apps are good. People are also looking for the next form factor. People want the glasses, the earbuds. What is your wish list in terms of having agents in your life?

SPEAKER_00

[SPEAKER_01] I started on that actually already, but then I was just getting bogged down by all the people using it and the daily grind. But if you're at home, I want to be in any room and at Star Trek when you say computer, I want to talk to my agent wherever I am and it should just be able to respond to me. It should know where I am. I have little iPads in every room and my agent can use the canvas feature and project stuff on those iPads. So if I ask a question that is easier to be answered by also showing me something, it could use the nearest display because it's aware of where I am. So the phone is just a very convenient input point, but I kind of want to talk to it from anywhere.

SPEAKER_00

Yeah. [SPEAKER_01] If I'm around and I have glasses, I should just be able to listen in and project something on me. But ubiquitous follow you everywhere. I think, yeah, once we have it's truly smart home. Yeah. [SPEAKER_01] Agents on your phone, but really you want ubiquitous agents and then you want maybe your upper case, open cloth, your private agent at work. You might have your lower case, open eye cloth. And then that cloth should be able to talk to your personal cloth in a way that both your company and you are comfortable with. So that's the future where we need to work out.

SPEAKER_00

Yeah. I just did a podcast with Mark Andreessen, who's a huge fan. And also have conversations with Andre Karpathy. Both of these guys are running open cloth to run their house. And I think open cloth for homes is underrated, but people are really discovering it. And my funniest irony is that it's only possible because the internet of shit means that most smart devices are terrible in security, which means open cloth can run them. [SPEAKER_01] Oh, it's going to be able to work so much better in a few months when the models are getting really good. Yeah, they're very good.

SPEAKER_00

Okay. One security question about prompt injection. How would you want to solve prompt injection? Or what ways in which have you been thinking about the prompt injection problem? [SPEAKER_01] Probably not enough yet. On the other hand, the front end models are really quite good at detecting all the cases where stuff randomly comes in from a website or an email is usually not a problem anymore.

SPEAKER_01

[SPEAKER_02] Your market is untrusted content. Very hard to exfiltrate you from that.

SPEAKER_01

If I have unlimited access to your claw, I can bombard it with stuff, then there's still a chance. You're going to find a way. Then there's still a chance, but for one of things, it's no longer the biggest problem. If you use that's also why you know this is probably the angle where some people say, oh, Peter doesn't like local models. But then I see people running a 20 billion parameter model that just does whatever you tell it and it's not trained to have anything to do. If you don't have any defenses at all, that's still problematic. If you run that and then you use a web browser or email, it would worry me. That's why OpenClob warns you if you use a small model. And then people spin the whole thing like, I hate small models. I love that we support everything, but you have to steer the regular user a little bit in a direction to make it harder for them to shoot themselves in the foot. Yeah, there are some ideas for prompt injections. It's a little bit away. I have more to announce there.

SPEAKER_01

[SPEAKER_00] I think Simon Willison has been working a lot on this. I mean, he coined the term prompt injection and the dual LLM approach seems smart. And I'm not smart enough to figure out all the ways in which it can be attacked. At some point, trust just has to be a thing, right? And I probably there's something interesting I found out from talking with Vincent, who is speaking next, is that you guys had to implement the same trust system that Toby Luca had to implement, which is you build reputation over time and things with more trust gets more privileged access, right? And I think that makes sense. That's part of the story. Yeah, yeah, yeah.

SPEAKER_01

[SPEAKER_00] Okay. So what was some broader questions? What cool projects would you like to work on once you have more free time? I mean, I wanted to work on dreaming. And I like my maintenance worked on dreaming while you were dreaming. You just shipped it, right?

SPEAKER_00

Yes, yes. What is dreaming?

SPEAKER_01

It's a way to reconcile memories and create a dream log. It goes through your session logs. We found out from the Anthropic source code leak that they're also working on dreaming, right? Oh, yeah, yeah. I mean, there's I'm pretty sure there's more companies working on that. But think about how do we learn as humans? You experience a lot of things during the day and then you sleep. And in sleep, your brain is garbage collecting, converts some locally stored memories into long-term storage and drops others. And that similar idea is that I think could also be very useful for agents. And what we shipped on dreaming is a first little step in that direction. Yeah.

SPEAKER_00

And it's related to the wiki thing that Andre has been talking about? Where you sort of collect everything into a... [SPEAKER_01] There's more companies working on that. [SPEAKER_01] But think about how do we learn as humans?

SPEAKER_01

You experience a lot of things during the day and then you sleep. And in sleep, your brain is a garbage collect, converts some locally stored memories into long-term storage and drops others. And that similar idea is something I think could also be very useful for agents. And what we shipped on dreaming is a first little step in that direction. Yeah.

SPEAKER_00

And it's related to the wiki thing that Andre has been talking about? Where you sort of collect everything into a... [SPEAKER_01] Wiki is more memory, but everything kind of blends a little bit together. [SPEAKER_01] The beauty of OpenClaw is that we can just try stuff. [SPEAKER_01] Everything we worked on for the last month or so is that in the beginning it was a big spaghetti code base mess. [SPEAKER_01] And now everything is an extension, a plugin. [SPEAKER_01] So you can replace memory.

SPEAKER_01

You can add the wiki. You can add dreaming.

SPEAKER_00

[SPEAKER_01] You can add whatever crazy idea you have and just make it your own. [SPEAKER_01] You don't have to send everything to a pull request because we are still completely overloaded on those. [SPEAKER_01] But it's more like Linux where you just can install your own parts. Yeah.

SPEAKER_01

[SPEAKER_00] And you are building what a lot of people think is the most consequential open source since Linux. [SPEAKER_00] How do you deal with that? [SPEAKER_00] How do you deal with the fame?

SPEAKER_02

[SPEAKER_00] What is a day in your life as the BDFL effectively of something like this? [SPEAKER_01] Well, there is still a lot of coding.

SPEAKER_01

There is also a lot of... By the way, in between sessions he was coding. Yeah. You get token anxiety. Something has to be running. You have to push the agents, right? Yeah. We have shifted a little bit. Now it is a lot more talking and steering people in the right direction. Because there is a lot of things that we have already learned in OpenClaw. So part of my role at OpenAI is to help them not make the same mistakes again. And at OpenClaw is to try out new things that seem exciting. And some might work and some might not work. Enable companies to build their own claw without having to fork away. But making everything more customizable. Yeah. And sometimes I sleep.

SPEAKER_01

[SPEAKER_00] Sometimes you sleep. [SPEAKER_00] Okay.

SPEAKER_01

[SPEAKER_00] Great. [SPEAKER_00] I think this is the last closing question.

SPEAKER_00

What skills do you want humans and engineers in particular to focus on developing in the age of AI? [SPEAKER_01] Taste was a big one but I already mentioned that. [SPEAKER_01] System design is still very important. [SPEAKER_01] Yes. [SPEAKER_01] We talked about this in San Francisco. [SPEAKER_01] Yeah. [SPEAKER_01] If you don't think about that you will eventually slide yourself into a corner. [SPEAKER_01] Right?

SPEAKER_01

Just by defining the boundaries. Everything is in the clinker but you still need to ask the right questions.

SPEAKER_00

[SPEAKER_01] Otherwise that makes the difference between good code that comes out or really bad code that comes out. [SPEAKER_01] And that's still where all the knowledge you have about how you build software you can apply to steer the agent into something that is not slop. [SPEAKER_01] Yeah.

SPEAKER_01

And then I think a skill that is becoming more and more important is saying no. And that's something I had to learn as well because even the wildest idea is just a prompt away. And usually this one idea is never the problem but this idea and this idea and this idea and this idea and this idea and then how all of that fits together. That's the problem.

SPEAKER_00

[SPEAKER_02] Yes. [SPEAKER_01] So I think we're still bottlenecked on thinking about big picture thinking.

SPEAKER_01

Yeah. Because imagine the world from your clanker. You're being thrown into a code base. You might have an outdated agent.md file but you basically don't know what this is. And then you tell me hey add user profiles. And you somehow add user profiles and connect it to the two things you see but you didn't see the whole system. Right. And that's where a lot of those localized solutions come from where the project has wards and it's our job to help the agent do its best work by providing them with hints. And you're going to say you want to consider this. You want to look there. How would this interplay with this.

SPEAKER_01

And then ultimately you get a system that actually is maintainable.

SPEAKER_00

Yeah. Well thank you for maintaining one of the most important software of all time and thank you for spending time with us.

SPEAKER_01

Thanks for having me. [SPEAKER_00] Hopefully you stick around and answer questions. Thanks. [SPEAKER_00] Thank you. What we worked on for the last month or so is that in the beginning it was a big spaghetti code base mess. And now like everything is an extension, a plugin. So you can replace memory. You can add the wiki. You can add dreaming. You can add... I don't know. Your... Whatever crazy idea you have and just make it your own. You don't have to send everything to a pull request because we are still completely overloaded on those. But it's more like Linux where you just can install your own parts.

SPEAKER_00

Yeah. Yeah. And you are building what a lot of people think is the most consequential open source since Linux. Which... I don't know. How do you deal with that? How do you deal with the fame? What is a day in your life as the BDFL effectively of something like this?

SPEAKER_01

Well, there is still a lot of coding. There is also a lot of... By the way, in between sessions he was coding. Back there. Yeah. You get token anxiety. You have to like... Something has to be running. You have to push the agents, right? Yeah.

SPEAKER_01

We have shifted a little bit. Now it is a lot more talking and steering people in the right direction. Because there is a lot of things that we have already learned in OpenClaw. So like part of my role at OpenAI is to like help them not make the same mistakes again. And then at OpenClaw is like try out new things that seem exciting. And some might work and some might not work. Enable companies to like build their own claw without having to fork away. But like making everything more customizable. Yeah. And sometimes I sleep.

SPEAKER_00

Sometimes you sleep. Okay. Great. I think maybe this is the last closing question. What skills do you want humans and engineers in particular to focus on developing in the age of AI?

SPEAKER_01

Okay. Taste was a big one but I already mentioned that.

SPEAKER_01

System design is still very important. Yes. We talked about this in San Francisco. Yeah. If you don't think about that you will eventually slide yourself into a corner. Right? Just by defining the boundaries. Like the funny thing is like everything is in the clinker but you still need to ask the right questions. Otherwise that makes this difference of like good code that comes out or like really bad code that comes out. And that's still where like all the knowledge you have like how you build software you can apply to steer the agent into something that is not slop. Yeah. And then I think a skill that is becoming more and more important is saying no.

SPEAKER_01

And that's something I had to learn as well because even the wildest idea is just a prompt away. And usually this one idea is never the problem but like this idea and this idea and this idea and this idea and this idea and then how all of that fits together. That's the problem.

SPEAKER_02

Yes.

SPEAKER_01

So like I think we're still bottlenecked on thinking and about like big picture thinking. Yeah. Because imagine the world from your clanker. Like you're being thrown into a code base. You might have an outdated agent.md file but you basically don't know what DF this is. And you like then like you tell me hey add user profiles. And you like somehow add user profiles and connect it to the two things you see but you didn't see the whole system. Right. And that's where a lot of those localized solutions comes where like the project has like wards and it's our job to like help the agent do its best work by like providing them with like hints.

SPEAKER_01

And you're going to say you want to consider this. You want to look there. How would this interplay with this. And then ultimately you get like a much a system that actually is maintainable.

SPEAKER_00

Yeah. Well thank you for maintaining one of the most important software of all time and thank you for spending time with us. Thanks for having me. Hopefully you stick around and answer questions. Thanks. Thank you. All right.