[SPEAKER_06] Welcome to the workshop.
SPEAKER_06
I am Jonas and this is Misha. We both work at iterate. And this is going to be chaos. I think pure chaos. First of all, because we only decided last Monday to do this. And then there was a bunch of personal stuff and half term and the kids were sick. And then this morning we came here and I thought this was going to be an hour and 20 hackathon or something. And then we came here and we're told you're going to do it without an audience because of some fire thing. And now there's an audience again, which is amazing. So we're going to try and do this as an improvised hackathon. But I've only just pushed the SDK literally one minute ago.
SPEAKER_06
And Misha is going to try to live fix it. But I hope it will still be worth your time because I think there's some, hopefully at least some interesting ideas. Or maybe one way to put it is I would to find out whether this is dumb or cool. And you can help me with that. So please also ask clarifying questions. Because part of this, the reason we said last week, okay, let's do this workshop is because these have been ideas that have been noodling around over the last year, but we haven't really actually done anything with. It's not really commercial value to it necessarily. But I think it's how I would like to build agent harnesses.
SPEAKER_06
So I've been noodling with this for a while now. And obviously you've got Claude and you've got Pi now. But back in the day you didn't. And so you make your own coding harness. And there's all these different ways to do it. And I think the way I would to do it is I would do it purely event sourced. And I say here, aka debuggable. And that is because everything kind of skirts around being event sourced, all these things, without quite being event sourced. There's always something that has a side effect that you later can't tell.
SPEAKER_06
And then you can only see it in the hotel traces or something, which doesn't really make sense to me because normally these agents, the way they work, and we'll see that in a second, is they already have a log of events, right? So if you just say everything that could possibly happen is in there, it'll be really easy to debug. And as we'll see, hopefully, a little bit strange, but potentially in other ways also really easy to extend. And then the second thing is I want this to be extensible. And we've seen this, I think, with Pi, which is just incredible. How valuable it is to make an agent extensible, not just by humans, but also extensible by itself.
SPEAKER_06
And one thing that I want in the extensibility is I want to be composable. I don't think we'll get there within one hour with all of you, but maybe afterwards, if you say, oh, this is actually cool and not dumb, we'll keep working on this. And you can say, oh, I made this extension, you made that extension, and you can combine them. And because I think we don't really yet know what is the best recipe for agents, basically, for agent harnesses. And it's been too hard to experiment, even in Pi. Everything has to run in a single thread, in one process over here on this particular computer, and so on and so forth. So I think we can maybe do a little bit better than that.
SPEAKER_06
Then I think it should be on the edge, publicly rootable. The way I think of it is basically an intelligent entity, a digital one, that's not a robot in the future. It's just internet-connected, basically, a server program, right? It just speaks HTTP.
SPEAKER_06
So the slides, broadly speaking, that should be sufficient for almost anything. And then I would just put these agents on the edge, speaking to the internet straight away. This is not very safe. None of what we're doing today is safe, by the way. There's no authentication you'll see. You'll all be able to see everybody else's event streams and agents and whatever else. You should not put any secrets in the actual payloads of the events or rotate them afterwards, right? So we can solve those things. But broadly speaking, the moment an agent exists, it should have a URL. That's just my opinion.
SPEAKER_06
Because otherwise, you end up making all these things and then you tie yourself backwards, inventing a connector concept just so you can get Slack messages in or God knows what. And I think that should be really easy. Or maybe a human should fill in a web form and the result of the web form should be input for the agent again or something. That's the same thing as a Slack webhook. And then it should be distributed. And this is potentially a double-edged sword, as we'll see, right?
SPEAKER_06
But by distributed, I mean you should be able to have an agent running on one computer here on the server and my plug-in running over here and your plug-in running over there and yours is written in Rust and mine is written in TypeScript and it all just doesn't really matter. And the reason that is bad is because you will eventually get race conditions and loops, right? Two plug-ins sending messages back and forth, creating an endless stream of events in your agent. But on the flip side, you have those issues normally anyway in most agent harnesses I've seen. So it's quite good if you can preempt that a little bit. [SPEAKER_07] Okay.
SPEAKER_06
So let's talk about this thing that we have invented. Oh my gosh. What is this? This isn't the same as in my file system. So I'll use the one in my file system. Basically, we have created this service over the last week called events.iterate.com. It has a web UI and we can use that. I think you'll certainly use it. But for now, we'll just play with it with curl because I think it makes it more tractable. In the agents app, we have this concept of agent paths. So just like a file in a file system, every agent has a path. What is this? This isn't the same as in my file system. So I'll use the one in my file system.
SPEAKER_06
We have created this service over the last week called events.iterate.com. It has a web UI and we can... You can use that. I think you'll certainly use it. But for now, we'll just play with it with curl because I think it makes it, in some sense, more tractable. In the agents... In the events app, we have this concept of agent paths.
SPEAKER_07
[SPEAKER_06] So just like a file in a file system, every agent has a path.
SPEAKER_06
That's a hierarchy. And you just have here... Let me show this here. You just have raw events. This is YAML formatted raw events in each of these paths. And that is the entire primitive that we're going to be building our agent on. And so we can start by playing with this. We can say, okay, we have a path prefix. If you copy this from the repo that Misha is going to put it in the readme in a second... I'll add workshop.md as well. [SPEAKER_07] Yeah. Yeah, okay. Oh, yeah. Maybe that's why. [SPEAKER_04] I added the readme, but not the workshop. Yeah. So let's set ourselves a path prefix here. In my case, that is just going to be Jonas Templestine or something.
SPEAKER_06
They start with forward slash, by the way, which is a little bit awkward, potentially. But the service is very tolerant of URL encoding your slashes or doing whatever you want. But I wanted good curl economics, and I also wanted them to start with a forward slash. So you have a well-defined route. Then for the base...
SPEAKER_07
[SPEAKER_04] It is now pushed to the repo.
SPEAKER_06
[SPEAKER_04] But have you shared the repo? Oh, yeah. Because now you could actually do this on your computer.
SPEAKER_04
[SPEAKER_06] Let's do that, actually.
SPEAKER_06
If you go to github.com slash iterate, there is an AI-engineer-workshop. And in there, you should be able to... If you click on this workshop.md file, that's what we're now going through. So you should be able to copy and paste the curls from there. And again, there's no authentication. So you can just do this. And in fact, why don't a couple... This is hitting our live deploy.
SPEAKER_04
Yeah, yeah, yeah. [SPEAKER_06] Eventstore.iterate.com.
SPEAKER_06
So... This is entirely a proof of concept of how I think this should work. Basically what we're going to do is we're going to turn this simple event stream into a fully-fledged agent without really deploying any server software or anything like that. Maybe, yeah, I should be able to see if anybody has managed to do it.
SPEAKER_06
I'll wait until two or three people have managed to successfully curl something. And then I'll carry on because I think it'll be good if you curl it yourself. So it's here in this AI-engineer workshop, workshop.md. If two people have the same name, that's going to cause problems. That doesn't matter. That's fine. Okay, there we go.
SPEAKER_04
[SPEAKER_06] Got a few people.
SPEAKER_06
Nice. Okay, so this is pretty good. By the way, you might see this here and there. There's also this thing down here called project slug. If you really start stepping on each other's toes, you can just type what, use a different project slug and it's a completely new database, a new namespace. You can pass that in as an HTTP header.
SPEAKER_06
If anybody needs that later, you can ask me. Okay, so now we have a few people curling things. I will also start curling things. So now I'm just going to say hello world here to my stream. And it will come back. [SPEAKER_06] And straight away we notice a few things. There's a basic event envelope shape. We have a type that I've just invented here.
SPEAKER_06
Events don't need to have anything but a type, but most of them also have a payload. Then there is the stream path, which came from the URL that we posted this to. [SPEAKER_06] And there's an offset. This is just an auto incrementing integer that starts at one. And so there's basically a point of serialization at the server that just increments them. And you get a created at. And then here what we can do with curl, which is quite nice. If you do dash n, you can do streaming SSE. So this here is literally nothing magical. This is literally just slash Jonas Templestine slash hello world. This is literally the curl that we're doing right here.
SPEAKER_06
And there I can see the messages coming through. Very nice. And I can add another message. And now all the messages will show up there. And there's also a, oh, if you do question mark live equals true, this is inspired by the durable stream standard, but slightly different. But if you do live equals true, the connection will stay open and it'll just keep sending you more events as you put them through. So this is sort of already an AI agent. If Misha was simultaneously giving responses to my hello world, this would already be an agent.
SPEAKER_06
And then there's a, my AI has told me you can, if you're going to be doing that a lot, you can pipe this through SED, get just the data elements, and then pipe it through jq, and then you get a little bit nicer formatting. So this is so trivial that I actually do spend a lot of time in curl, just curling it. Then what can you do? The thing is very tolerant, in general, of stuff you post to it. And this will become relevant if you wanted to say, send web hooks from random third parties there. My theory is always that the LLMs are good enough. They can basically just consume raw web hooks from anywhere and they'll roughly do the right thing.
SPEAKER_06
So what specifically the server will do is, if you send it something that doesn't validate, hogwash here, it says there's no type property, so there's not actually a valid event. Then you can see here what it has appended is an error.
SPEAKER_06
But because we're event sourcing, the error is also just an event. And it just says here, okay, you have an invalid event appended event. And this will become relevant if you wanted to send web hooks from random third parties there. My theory is always that the LLMs are good enough. They can consume raw web hooks from anywhere and they'll roughly do the right thing. So what specifically the server will do is, if you send it something that doesn't validate, hogwash here, it says there's no type property, so there's not actually a valid event. Then you can see here what it has appended is an error. But because we're event sourcing, the error is also just an event.
SPEAKER_06
And it just says here, okay, you have an invalid event appended event. Another thing that we will notice is that the type here that our server has created, it starts with HTTPS events.iterate.com. You don't have to do that, but that is something that we like to do because that literally leads to the documentation for that event type. And if it's going to be an opaque string, why not make it something like that? But obviously you can just do single words or two words or whatever. But you will see these URLs and they're just types. Then what else is interesting that we can do?
SPEAKER_06
This is boring, but again, if you're going to do webhooks, you can put an item potency key in, which just means if you have the same item potency key twice, you don't get the event appended twice. You can pause a stream. I think in the interest of time we can skip that. You can resume a stream. The reason that pausing is important and we put this in even though we literally just made this a couple days ago is because of the infinite loops. It's very easy to accidentally put thousands and thousands of events in an infinite loop. And so if you put more than 100 events in a second or something, it is just going to circuit break.
SPEAKER_06
And then the stream will be paused until somebody unpauses it. So basically if you try to... Here, actually, let's do that. Because again, for those of you who haven't seen event sourcing, this might be a bit weird. How do you pause a stream? Well, you pause a stream by... Whoops, this is the wrong one. You pause a stream by appending an event that says you're paused. The only external interface to the agent is... Now it's complaining. [SPEAKER_07] Why is it complaining? Ah, because I think there needs to be a reason or something, maybe. Yeah. Reason demo. Yeah, there you go. So now this thing is paused.
SPEAKER_06
So now if I try to send it the hello world event from earlier, it's just going to barf. And it'd say, no, no more events allowed. And that is one thing that doesn't appear as an error in the stream because otherwise you would still get infinite numbers of error events. What it'll do is it'll just observe certain events, rewrite them a little bit, and make a new event out of it. But I think that's a bit advanced, so we'll do it later. It can also do weird things like it can... You can schedule things. Like some agent SDKs you'll know can do this. But here, if you want a heartbeat event every five seconds... [SPEAKER_07] Oh. [SPEAKER_07] Oh, God.
SPEAKER_06
[SPEAKER_07] Now I need to resume the stream because it's paused.
SPEAKER_07
Let's see.
SPEAKER_06
[SPEAKER_07] Stream resumed.
SPEAKER_06
[SPEAKER_06] Okay. So now I am just creating this heartbeat event every five seconds, which can be quite handy.
SPEAKER_06
Or you can schedule something for 10 minutes in the future or at a certain time. And again, it's just an event that says do this thing in the future. I think I will skip over these, but you can cancel the schedule. You can tell the stream by appending an event to send your server or some third-party server or your Slack API endpoint or whatever an event every time an event gets added. So you can basically subscribe to the stream. Here we're subscribing in our terminal. We're doing a pull subscription, right?
SPEAKER_06
We're connecting to an HTTP server and we're getting... We're pulling... We're the HTTP client and they're the HTTP server and we're getting an SSE response stream of streaming events. But it can also do the other way around where it will push you the stuff that gets appended or push you a filtered stream. That we're gonna be using for the next 45. [SPEAKER_04] But questions have to be submitted as an event to the question stream. [SPEAKER_04] Yeah, exactly. Well, that can be the queue. Does... I assume lots of things don't make sense. So please do ask them because I think this will be extremely random and strange so far.
SPEAKER_06
[SPEAKER_04] What does Iterweight do with your company?
SPEAKER_07
[SPEAKER_06] Oh, we're a hacker hobby club at this point, to be quite honest. [SPEAKER_06] We have not got a launch product. [SPEAKER_06] But if we did, it would be, I think, based on this architecture. [SPEAKER_06] Anyway, I can talk about it afterwards. [SPEAKER_04] Okay, okay, okay. [SPEAKER_06] Yeah. [SPEAKER_06] It's more homebrew hacker club than commercial enterprise at the minute.
SPEAKER_06
But it'll get there. Yeah. [SPEAKER_02] I have a question. [SPEAKER_02] So, I know that we'll be building event-based agents. [SPEAKER_06] Yeah. [SPEAKER_02] Any... Maybe that's a basic question, but agents that are on the market right now, aren't they event-based?
SPEAKER_06
[SPEAKER_02] What's the... It's just the API. If you have tried to make an open code extension or plugin or a pie extension or a Claude extension, they all have an API surface that is a little bit bigger than what you would have or what we're gonna have in a minute. It's not by much, right? Yeah. [SPEAKER_02] Any... [SPEAKER_02] Maybe that's a basic question, but agents that are on the market right now, aren't they event-based? [SPEAKER_02] What's the... It's just... I think it's just the API... If you...
SPEAKER_06
Have you tried to make an open code extension or plugin or a pie extension or a Claude extension? And they all have an API surface that is a little bit bigger than what you would have or what we're gonna have in a minute.
SPEAKER_04
[SPEAKER_06] It's not like by much, right? [SPEAKER_06] But maybe the streaming chunks for partial completions are a different kind of thing for some reason, rather than just an event of type, I've received, five letters of a streaming response. [SPEAKER_06] To me, that's a perfectly valid event. [SPEAKER_06] And the benefit then is that you can...
SPEAKER_06
I'm a one-abstraction kind of guy, right?
SPEAKER_06
This should be a web standards base. This should be... There should only be one thing, which is an event. And then as you'll see, the thing that was at the top of this file, I think you can build a super expressive state-of-the-art coding agent by just implementing a stream processor that reduces over the event stream and occasionally executes side effects.
SPEAKER_04
[SPEAKER_06] That's all it does, right?
SPEAKER_06
It just sits there and says, okay... I'll show you that in a second. It's a bit weird, I would say. A weird way of programming. [SPEAKER_05] So if I understand correctly, you're saying you take events that you get generated from an AI model, and then you're basically going to pipe them through this stream processor. [SPEAKER_05] Is that the idea? So the stream processor is...
SPEAKER_04
[SPEAKER_06] Or the events that iterate the app is the storage.
SPEAKER_06
You can almost think of it as a database. You know, Convex is a sort of... They kind of give you the illusion that they're running TypeScript in the database. You can almost think of this... This could be a programmable stream or something, right?
SPEAKER_02
[SPEAKER_06] Because actually one thing that you can also do, we'll see later, once you hack on your stream processors, you can also make an event that contains the source code of your stream processor. [SPEAKER_06] And then it'll just run on every event.
SPEAKER_06
And so it's basically...
SPEAKER_02
[SPEAKER_06] It is the... [SPEAKER_06] I think you can experiment with a huge array of different kinds of agent harness implementations that can respond to real events on the internet and make real API requests without deploying your service or something like that. [SPEAKER_06] All you need is this sort of streaming service that has some of these things. [SPEAKER_06] Like, specifically, it needs to be able to wake up other APIs and programs and so on.
SPEAKER_06
I don't know what that... Yeah. That... Does that answer your question at all? [SPEAKER_05] Yeah, I think so, yeah. Okay. So we try and make now a... A slightly more serious version of these curls. So... Does anybody here not know TypeScript? I think... So I think one of the arguments in favor of this architecture that I also put in that... In that... In the blurb, I think, is this is, in principle, polyglot, right? And there is a... In any language, there is actually at events.iterate.com slash API slash docs. You have just... This is an exceedingly simple API, right? You can make yourself... There is an open API spec. You can make yourself a client and so on.
SPEAKER_06
But for the purpose of this workshop here, we'll just do a TypeScript version. And we've shipped a client in an NPM package already. But I just wanted to point out, even though we're doing everything in TypeScript, the whole point is that we wouldn't have to. Right?
SPEAKER_05
[SPEAKER_06] But we just have to do one thing for this workshop. [SPEAKER_06] Okay. [SPEAKER_06] So let's do something pretty simple.
SPEAKER_06
Is the AI engineer workshop package now up to date and working? I think it is, right? [SPEAKER_04] I think so, yeah. [SPEAKER_04] You might need to pull your repo because there was a client-side type error. In the iterate repo? [SPEAKER_04] No, in the AI engineer workshop. [SPEAKER_04] Okay, that's fine. [SPEAKER_04] It's in the example.ts. Yeah, but I'm... Oh, maybe I will actually use that. Then I'll be able to experience the same bugs if there are any. Yeah, that's probably better. Okay. Okay. So I think if you wanted to just have a quick start, you can just clone the AI engineer workshop repo. There's already a teeny tiny example file in there.
SPEAKER_06
And then we'll just add a couple more as we go along. And then workshop.md has runnable curl as well. Nice.
SPEAKER_05
[SPEAKER_06] Yeah.
SPEAKER_06
That's fine. If you wanted, you could put a couple of the follow-on examples that are in the iterate repo that have actual processors. [SPEAKER_07] Yeah. [SPEAKER_07] I can take that. [SPEAKER_07] I don't know what's going on here. [SPEAKER_07] Aha. There we go. Hmm. My computer seems to have slowed to a crawl trying to open a cursor window. This has been, unfortunately, a somewhat regular occurrence recently. Oh, no. I think this might be what you mentioned about the internet. I think it's just struggling because it can't open a terminal because my terminal tries to connect with the internet. [SPEAKER_07] Oh, well. [SPEAKER_07] Then let's do this.
SPEAKER_06
[SPEAKER_07] That's not the internet. [SPEAKER_07] What is it? [SPEAKER_07] There we go. [SPEAKER_07] Okay. Okay. So we have here packaged up an SDK here to this AI Engineer Workshop package that you should be able to just pnpm install or you just use this example here. And then basically, what do we need in our curl? Right? We needed a base URL, which is just events.iterate.com. That's where our server is.
SPEAKER_04
[SPEAKER_06] We can use a path prefix. [SPEAKER_07] Oh, well.
SPEAKER_06
[SPEAKER_07] Then let's do this.
SPEAKER_04
[SPEAKER_07] That's not the internet. [SPEAKER_07] What is it? [SPEAKER_07] There we go.
SPEAKER_06
[SPEAKER_07] Okay. Okay. So we have here, we have packaged up an SDK here to this AI Engineer Workshop package that you should be able to just pnpm install or you just use this example here. And then what do we need in our curl? Right? We needed a base URL, which is just events.iterate.com. That's where our server is. We can use a path prefix. And then we can run this function here called createEventsClient. That is from our SDK. And then this is under the hood. This is... Hmm. [SPEAKER_07] I really don't know what's going on with this terminal.
SPEAKER_07
It's very strange. Hmm.
SPEAKER_07
Hmm.
SPEAKER_07
[SPEAKER_06] Okay.
SPEAKER_06
Maybe everything is just incredibly slow.
SPEAKER_06
Do you want to try mine? [SPEAKER_06] Okay. Maybe, but there's nothing actually hogging the CPU. I do think it might be actually the network, but we'll see.
[SPEAKER_06] So here.
SPEAKER_06
So here you have... I don't think this has picked up. Anyway.
SPEAKER_07
[SPEAKER_06] Okay. [SPEAKER_06] So this is what we were doing just now, right?
SPEAKER_07
[SPEAKER_06] We say, okay, we're making ourselves a client. [SPEAKER_06] And then here there is a function called stream. [SPEAKER_06] This is the right-hand side of the terminal that we had earlier. [SPEAKER_06] And then you say live true and given the stream path.
SPEAKER_06
And we can say, okay, if the event is not of type ping, we will do nothing. But if the event is of type ping, then we will... This is a completely unnecessary helper down there. Then we can just do client.append. I don't understand why the TypeScript language server isn't picking up any of the types for me, but I can't tell if that's just... Cursor being weird again. You're an example, then.
SPEAKER_06
Yeah. Anyway. [SPEAKER_04] Good. [SPEAKER_04] Quickly switch IDs. [SPEAKER_04] That's fine. [SPEAKER_04] It is working for me.
SPEAKER_07
[SPEAKER_06] Okay. [SPEAKER_06] Yeah.
SPEAKER_07
[SPEAKER_06] I think this is unfortunately a cursor issue. [SPEAKER_06] Okay.
SPEAKER_06
And then you should be able to just... If you have a modern version of Node, you should be able to just...
SPEAKER_06
run... Or maybe not. Did I make it... It says invalid TypeScript syntax. You know what?
SPEAKER_06
I'm just going to restart cursor. I think that is worth it. Is anyone able to run this example TS on their computer? There you go. I am. Hmm. Okay. And are you able to send a ping to it and get a pong out of it? Okay. Very cool. [SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that?
SPEAKER_04
[SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that? [SPEAKER_07] How are you able to do that?
SPEAKER_06
[SPEAKER_07] How are you able to do that? I wonder if we use, I don't know if you can see this, but all my commands, they have a several second delay. [SPEAKER_04] Yeah, it's looking okay to me if you want to just swap laptops. [SPEAKER_07] Maybe. Yeah, no, I know, but it's just my editor doesn't work. And I also have several non-responsive cursor. So this is unfortunately a pattern with cursor recently. [SPEAKER_04] If you want to swap laptops, it does seem to be working okay online. I would just give everybody 60 more seconds to just get the basic demo working and then...
SPEAKER_06
[SPEAKER_07] Just let me know. [SPEAKER_07] Yeah, because I think this is still almost an hour to go. [SPEAKER_07] It's not good to use somebody else's computer. [SPEAKER_07] So...
SPEAKER_06
Maybe I will continue on yours and I'll just restart my computer. I think that's a good idea because we can... Yeah. [SPEAKER_04] I also have examples 01 and 02. [SPEAKER_04] Oh, really?
[SPEAKER_04] Brilliant. [SPEAKER_04] I think working with some... [SPEAKER_04] What's your keyboard layout? [SPEAKER_04] Probably the same as yours. [SPEAKER_07] One can hope. [SPEAKER_07] Okay. [SPEAKER_07] Entire screen. [SPEAKER_07] Looks like... [SPEAKER_07] Looks pretty good so far. [SPEAKER_07] One of you got a terminal. [SPEAKER_07] Node examples 01 hello world. [SPEAKER_06] Perfect. [SPEAKER_06] Yeah. [SPEAKER_06] This works flawlessly. [SPEAKER_06] Has everybody got that more or less working? [SPEAKER_06] Because then we can just make an open AI coding agent now. [SPEAKER_06] I think that's all we need. [SPEAKER_07] Let's... [SPEAKER_07] Maybe...
SPEAKER_07
[SPEAKER_07] I'll ask... [SPEAKER_07] Do you have a speech control in here? [SPEAKER_07] Yeah. [SPEAKER_04] It's... [SPEAKER_04] Control and option. [SPEAKER_04] It's kits one...
SPEAKER_06
[SPEAKER_07] Yeah. [SPEAKER_04] And then?
SPEAKER_04
That's it.
SPEAKER_07
[SPEAKER_04] It's recording.
SPEAKER_06
[SPEAKER_07] Hello. [SPEAKER_07] Oh. [SPEAKER_07] Cool.
SPEAKER_04
[SPEAKER_07] See that red bar?
SPEAKER_06
[SPEAKER_07] It's...
SPEAKER_07
[SPEAKER_06] Hey. [SPEAKER_06] I would like to make a third example called agent processor. [SPEAKER_06] And for that, we need to install just the normal open AI SDK. [SPEAKER_06] Please do that and just make me a new file that looks like the hello world processor.
[SPEAKER_06] Okay.
SPEAKER_07
[SPEAKER_06] So I just gotta gather everybody again after I left you for about 10 minutes. [SPEAKER_06] Here, this is a normal pretty normal spaghetti code TypeScript program. [SPEAKER_06] Right?
SPEAKER_06
[SPEAKER_06] We've done here is we're doing some boilerplate work. [SPEAKER_07] Cool. [SPEAKER_07] See that red bar? [SPEAKER_07] It's...
SPEAKER_04
[SPEAKER_06] Hey. [SPEAKER_06] I would like to make a third example called agent processor. [SPEAKER_06] And for that, we need to install just the normal open AI SDK. [SPEAKER_06] Please do that and just make me a new file that looks like the hello world processor. [SPEAKER_06] Okay. [SPEAKER_06] So...
SPEAKER_07
[SPEAKER_06] I just gotta gather everybody again after I left you for about 10 minutes. [SPEAKER_06] Here, this is a normal pretty normal spaghetti code TypeScript program. [SPEAKER_06] Right? [SPEAKER_06] What we've done here is we're doing some boilerplate work. [SPEAKER_06] Here, if we want to subscribe, we have to make a subscription.
SPEAKER_07
[SPEAKER_06] Oh my gosh. [SPEAKER_06] This example that you put in there has an abort signal and handles all kinds of things.
[SPEAKER_06] And... [SPEAKER_06] Anyway, the... [SPEAKER_06] I think... [SPEAKER_06] The way I think about what we're really doing is we're writing a stream processor. And if you've never done stream processing, this might be a bit weird and people use slightly different names for these things. But really, what we're saying is there exists some kind of world state that we're interested in that can be derived from the events. In this example here that Misha made, we are trying to count the number of times a hello world event has been encountered in the stream. And then you can write the entire logic of your program as effectively a reduce function.
SPEAKER_06
This is synchronous. And it just looks at all the new events as they come in. And then it updates the state. That's all it does.
SPEAKER_07
[SPEAKER_06] So it takes a state and event. [SPEAKER_06] And it returns the new state. [SPEAKER_06] Or you can just also return nothing. [SPEAKER_06] It'll be fine. [SPEAKER_06] It'll just ignore it.
SPEAKER_04
[SPEAKER_06] And then you should not do side effects in this reduce function. [SPEAKER_06] And the reason I've split the reduce function from after append where you should do your side effects is because if you think about what happens if your program goes to sleep. [SPEAKER_06] Right?
SPEAKER_07
[SPEAKER_06] You're closing your laptop.
SPEAKER_04
[SPEAKER_06] At the moment, everything's running on our laptops. [SPEAKER_06] 100 new events go into the stream. [SPEAKER_06] Some of these events might in the future need to trigger LLM requests or something like this.
SPEAKER_07
[SPEAKER_06] And then you open your computer again. [SPEAKER_06] You start your program again. [SPEAKER_06] And you don't want your program to go and make a ton of LLM requests for all of those 100 events in the past necessarily. [SPEAKER_06] Instead, you want it to catch up all the way and then decide what to do with the state at the end. [SPEAKER_06] And so we've made this super lightweight abstraction here of a processor that basically gets hooked up into a processor runtime.
SPEAKER_06
And then what it does is it splits your work into reducing events into state. And then if you want to do some side effects, like for example, appending to the same stream again, you can do this. By the way, I think you can also do this, which is potentially pretty interesting. You can literally just append to a child path or you can append to a parent path or whatever else you want. It's basically meant to be a file system.
SPEAKER_06
And so if you wanted to build sub agents on this thing, all you would do is probably you would just append to dot slash Boris and pretend to Boris as a sub agent.
SPEAKER_06
And then you would just subscribe to some special events that Boris might output like I have the result. And then you would automatically be woken up again on the parent when Boris has the result. And is this confirmed, Misha, that this runs? How do you run this? This just exports the—you need the runtime, right? [SPEAKER_04] Yes, there is a CLI command that I now can't remember because my agent has been running it. [SPEAKER_04] Where you can, you should be able to do pnpmw and then file. Did you do all that just now? I did that about a week ago, but you may have changed. No, no, no. Yeah. Okay. So okay. Okay.
SPEAKER_06
Let's go to our repo. Like I promised, this is a little bit. These are the, so you can actually also do this. If you go to iterate slash iterate AI engineer workshop, here there are a bunch of examples of files that are... Workshop 2. I was going from workshop 2. Yeah, I also... [SPEAKER_07] These out of date ones. That was meant to be renamed. Ah, here. This is basically the snippet we want. Okay. There is this, there's this thing that you can have. [SPEAKER_07] So here now we've just made a processor, right? This is just an inert thing. [SPEAKER_07] And then there's a thing called a pull subscriptions processor runtime.
SPEAKER_06
And all that does is it takes your processor that you just made and it runs it. And then... I think you probably need to assign it to a variable because you'll just export default above. Yes. Doot, doot, doot. Mm-hmm. Okay. And here, let's just say... Just see if we can run this. Choo-choo. [SPEAKER_07] Sorry, question. [SPEAKER_07] Should we have access to these examples as we're showing? [SPEAKER_04] Yes, but unfortunately they need to be modified in order to do anything interesting. [SPEAKER_07] Okay, so those two are on the... [SPEAKER_07] Yes, those ones are, were added in the last few minutes, but... [SPEAKER_07] Yeah.
SPEAKER_06
[SPEAKER_07] I gave slightly ad copying instructions. [SPEAKER_07] But I think... It goes on committed, yeah. Yeah. [SPEAKER_02] I think that we're just going to have to slowly build it up in this repo and then push it once it works here, unfortunately. [SPEAKER_02] So here, basically what I've now done is I've taken this processor and I've hooked it up to this [SPEAKER_04] runtime and all the runtime does is it's just this boilerplate for consuming the stream [SPEAKER_04] and running the reducer and running the after append hook.
SPEAKER_06
[SPEAKER_04] And what we should see is once you submit hello world into this path here that is just Jonas slash example, you should get a response there. So if I go here in the UI and I go to Jonas slash example, and I think I said this should be event of time hello world. Oh no, this is up here. If all works, this would be the proof point that lets us, but it doesn't. Hello world scene. [SPEAKER_02] So here, what I've now done is I've taken this processor and I've hooked it up to this
SPEAKER_04
runtime and all the runtime does is it's just this boilerplate for consuming the stream and running the reducer and running the after append hook. And what we should see is once you submit hello world into this path here that is just Jonas slash example,
SPEAKER_06
you should get a response there. So if I go here in the UI and I go to Jonas slash example, and I think I said this should be event of time hello world. Oh no, this is up here.
SPEAKER_06
If all works, this would be the proof point that lets us, but it doesn't. Hello world scene. I don't remember if this example relies on the machine username, but if it does, then that would be mmcal not Jonas. Maybe.
SPEAKER_07
[SPEAKER_06] Let's just do this.
SPEAKER_06
Sorry, this is the worst workshop.
SPEAKER_06
Now I'm also on the wrong computer. It does after append.
SPEAKER_07
[SPEAKER_06] And. [SPEAKER_06] Hmm.
SPEAKER_06
Okay. Is my computer back up? [SPEAKER_04] Yes, but I don't know your password. [SPEAKER_07] Okay.
SPEAKER_06
[SPEAKER_07] Yeah, I'll do that in a second. Okay. This example here is not working.
SPEAKER_06
[SPEAKER_07] Okay.
SPEAKER_06
[SPEAKER_07] So I think we have to do plan B, but we might not actually be able to run this.
SPEAKER_06
[SPEAKER_07] Are you getting?
SPEAKER_06
[SPEAKER_07] Getting.
SPEAKER_07
[SPEAKER_06] I'm just going to walk through the code as we would have written it from scratch.
SPEAKER_04
[SPEAKER_06] Sorry about that.
SPEAKER_07
[SPEAKER_06] Because I think it illustrates the progression of things. And maybe if you. It's installing dependencies, which is slow. So.
SPEAKER_07
[SPEAKER_06] Yeah, but it's okay, exactly.
[SPEAKER_07] Eventually this will. [SPEAKER_07] It might be runnable by the end of the walkthrough. [SPEAKER_06] Exactly. [SPEAKER_06] So in this paradigm, as we've been using it in our agent, when you want to implement some sort of feature, what you do is you say, okay, what is the state I actually need in order to do my thing.
SPEAKER_06
[SPEAKER_04] In this case, we want to do an LLM request whenever somebody sends a message and says hi.
SPEAKER_02
[SPEAKER_04] So we have here in our user interface, and you can access this as well.
SPEAKER_04
There is a cheat code that makes it a bit easier to create new events of a type called agent input added. [SPEAKER_06] And this is deliberately the simplest possible thing you can imagine. [SPEAKER_06] This is basically just a string.
SPEAKER_06
And so what we're trying to achieve is that whenever somebody puts an agent input added event into an event stream, we want to react to that by making an LLM request. And for simplicity, we're just going to do it using the OpenAI SDK. So the OpenAI SDK, the way it works is you have to, at some point, before you want to use it, make yourself a new OpenAI client.
SPEAKER_06
And then you can use their responses API. So you have OpenAI.responses, responses.create.
SPEAKER_06
And then, in order to create it, you need some information, specifically you need a model state string. And so immediately there it's saying, okay, so let's just say our state for our LLM agent has a model string. Because that is one of the things that we will need when we come to actually make an LLM request, and there is nothing but event sourcing.
SPEAKER_06
So let's just say, okay, we have a model string, and that comes from our state. And then, you could easily imagine, in the future, this was one of the things that you could have done as an exercise in the hackathon, is you could say, let's add an event that is like LLM changed, LLM model changed.
SPEAKER_06
And then you have the tiny one line reducer, and it just sets the model to be a different model.
SPEAKER_06
And then, off you go, you make the next LLM request with a different model.
SPEAKER_06
And the other thing we need is instructions. This is, I think, a bit of a misnomer.
SPEAKER_06
So I would call it system prompt in our state.
SPEAKER_06
And I would say, okay, the system prompt for our LLM, what it's meant to be doing, we'll take that in our state.
SPEAKER_04
[SPEAKER_06] And then you need a history, a state history.
SPEAKER_07
[SPEAKER_06] And so we've defined this. [SPEAKER_06] And normally what you'll do is you'll just put this into a little TypeScript type, if you're using TypeScript.
SPEAKER_06
So here we're saying, this is the state of our agent, right? And in the beginning, it's really simple, but in the future, you can very easily imagine how you quickly add things to it.
SPEAKER_07
[SPEAKER_06] Like state, am I currently compacting? [SPEAKER_06] Or is a tool call currently in progress? [SPEAKER_06] Or whatever else you might have. [SPEAKER_06] And the idea is to create a system in which it is trivially easy with three, four lines of code to add one of those capabilities to it.
SPEAKER_06
And then you say, okay, what's my initial state? Because we're processing the stream. Initially, there's no events, so we need to know what we're initially starting with.
SPEAKER_07
[SPEAKER_06] And so we're just saying, this is, we're starting with an empty history and this system prompt and GPT 5.4. [SPEAKER_06] And then the next thing you do once you've done that is you need to decide what events do I need to derive this state. [SPEAKER_06] And in this case, we will just create a new event here.
SPEAKER_06
A new, we're using Zod here for schema.
SPEAKER_07
[SPEAKER_06] As we're saying, it's an agent input added event and it just has a string. [SPEAKER_06] And then whenever anything comes back from OpenAI, because that is a fact that has occurred, even if it's just a tiny fragment of a string or something else, we store that in an event.
SPEAKER_06
And that's it. These are the two main event types. I don't think we need any of this stuff here actually. Basically, what happened, as is often the case, I attempted to instruct an agent to rewrite all of these examples to be simpler to execute just before.
SPEAKER_04
[SPEAKER_06] As we're saying, it's an agent input added event and it just has a string. [SPEAKER_06] And then whenever anything comes back from OpenAI, because that is a fact that has occurred, even if it's just a tiny fragment of a string or something else, we store that in an event. [SPEAKER_06] And that's it.
SPEAKER_06
These are the two main event types. I don't think we need any of this stuff here actually. What happened is, as often the case, I attempted to instruct an agent to rewrite all of these examples to be simpler to execute just before, and it just rewrote them quite significantly. But this is really all you need. And then you have to write your reducer first. And the reducer is where you just say, okay, when an event is encountered, how do I update my state? And you say, okay, when an agent input added event is encountered, this here is using a Zod helper library. Actually, one that Misha made called Ski Match. It just gives you safety on this payload here.
SPEAKER_06
When I have an agent input added event, then I want my state to be updated as the current state plus the history, except the history also gets this little thing here appended. The reason we're doing that is because OpenAI's responses API, since we're making OpenAI processor, we're basically translating between our view of the world. We just have this agent input and it's a string, but OpenAI wants you to put role user and then put the string in the content field. So we're putting that there.
SPEAKER_06
And then the other thing we need to do, when you use the OpenAI streaming API, it'll chuck out tons and tons of these output response events. And one of them finally at the very end will be of type an end event with the end, yeah? It basically does what we really want here is in... Yeah, we did a little bit better, so... Oh, thank you. Is that actually right? No. [SPEAKER_07] That's just... Yeah. [SPEAKER_07] I don't suppose you have managed to get the... I have pushed workshop two to the... Yeah, but it just doesn't... ...to the demo repo, but it's more or less as is. I have a question, is it going to be streamed in? Like, so... Yes. Pushing the stream...
SPEAKER_06
Yeah, let's just admit defeat on the thing that I was going to do, and just try and show you how it works. Because we can get something very simple working, right? Because we have the extremely simple ping-pong processor that does work. And so here we can even just... [SPEAKER_04] Mmm... [SPEAKER_04] Um... [SPEAKER_04] Let's maybe actually start here. [SPEAKER_03] And then I hit insufficient quote, running my... [SPEAKER_03] Hmm... Joop, joop, joop. Yeah, we do now have the same examples on the other repo, but it'll... Yeah. Ugh!
SPEAKER_06
To be honest, I think we might have to just admit defeat, and we can just talk about something else, or give you the time back. Just because the problem is we can't actually run these things. And I think it would take me probably ten minutes of actually concentrating to get them runnable. So maybe we should admit defeat on that. [SPEAKER_07] Um, I do wonder if there's, does it sort of make sense what the idea is, especially with it being distributed? [SPEAKER_04] So right, that you can say, okay, I have my processor over here, and your agent can use it, and it can be on a different server, or it can actually be in a dynamic worker.
SPEAKER_06
[SPEAKER_04] Yeah. Um, I think without showing it, it doesn't really make sense. [SPEAKER_04] So, if the idea is, with the stream processing, so you take this event stream from OpenAI, and it contains events like output item added, or I know the unpropleted model, but I don't know if it's, but you've got some. [SPEAKER_04] It's very similar. [SPEAKER_04] And then you've got some series of text deltas, so. Yeah.
SPEAKER_06
So, the reducer here is squashing all of those, and then materializing that partial response into an event, right? So then, the bit I was trying to understand was with this stream processor framework that you built, these materialized events, are those things persisted as new messages on the stream? Um. Do clients consume the raw pens, and you're doing the materialization client side, or is this part of something that's happening in the stream? So, every, yeah, so everything that happens in the system, every streaming chunk, everything becomes an event in the stream. Right. And then you have reducers running in many different places.
SPEAKER_06
[SPEAKER_05] Like, for instance, even this thing with, is the stream paused, or is the stream not paused? The actual implementation of that is maybe that's actually useful, because this is running in production now. Like, there is a circuit breaker processor, and this is how the whole thing is built, even inside the event service. And the circuit breaker processor has an initial state of paused, false, and it has paused, reason, null. And basically, then it has a simple reducer that just checks what are the last hundred timestamps of events I've seen. If the last hundred timestamps I've seen don't go, you know, don't go more than a second back, then I'm just going to append an event.
SPEAKER_06
[SPEAKER_05] The actual implementation of that is maybe that's actually useful, because this is running in production now. [SPEAKER_05] Right, there is a circuit breaker processor, and this is how the whole thing is built, even inside the event service. [SPEAKER_05] And the circuit breaker processor has an initial state of paused, false, and it has paused, reason, null. [SPEAKER_05] And then it has a really simple reducer that just checks what are the last hundred timestamps of events I've seen.
SPEAKER_06
[SPEAKER_05] If the last hundred timestamps I've seen don't go more than a second back, then I'm just gonna append an event. [SPEAKER_05] So the reducer basically just accumulates these timestamps, and then the after append function here says, okay, if I have seen too many timestamps in the last second, I'm gonna throw the circuit breaker. [SPEAKER_05] So this is a real processor that runs inside my service. But let me show you another one. This one here, this UI, this is also a stream processor. What does this UI do?
SPEAKER_06
The UI takes all of these events, and for some of the events that it deems interesting, it chucks them, it basically projects them onto what we call feed items. So these things here that have a slightly nicer rendering, they are feed items, and they are not the same as events, but they're derived from events. So when you're making a UI for this thing, you're also just reducing.
SPEAKER_06
And because the reducer is just a synchronous function, right, if you knew, for example, I have a pie agent harness, but I'm just gonna do it. This is a processor, and it comes with a whole bunch of event types, and it has a reducer. You can just, in your processor, use their reducer.
SPEAKER_06
It's practically free, right?
SPEAKER_06
You can just import it and run it.
SPEAKER_07
[SPEAKER_06] And then you can just start building abstractions that rely on their event types. [SPEAKER_06] And so I think what I'll commit to doing, because this was such a shitty experience, is I will just do the whole rundown in video form on the internet.
SPEAKER_06
And that was one of the things I was gonna show. You can just hook up pi or clod or open code as well. And just say, okay, they come with a bunch of events, and you can store them in this event stream, and you can react to them or build on them.
SPEAKER_07
[SPEAKER_06] Does that answer the question about where the reducer runs? [SPEAKER_06] And then one of the things you can do with this is you can also actually just append an event that has some source code in it. [SPEAKER_06] And then it runs that particular processor literally automatically for you, horizontally scalable. [SPEAKER_06] Yeah. [SPEAKER_06] Do you have the example of showing how you do that where you're appending a function? [SPEAKER_06] Yeah, I do have that. [SPEAKER_06] I do have that. [SPEAKER_06] Even if it's not runnable, I think the theory is somewhat clear. [SPEAKER_06] I do have that. [SPEAKER_06] Yeah, actually we can do it here.
[SPEAKER_06] Yeah, I think maybe that is good. [SPEAKER_06] Instead of trying to write code, we can focus on the things that might work. [SPEAKER_06] So let's create a new stream. [SPEAKER_06] By the way, all streams are created completely implicitly. [SPEAKER_06] So you can just post to anything under the streams API and it'll make you a new stream. [SPEAKER_04] And then here there's in the UI and you can also play with this if you want to just see, there's a bunch of preset events. [SPEAKER_06] And one of them is here. [SPEAKER_06] You can append an event of type dynamic worker configured to this thing. [SPEAKER_04] And what does it have?
[SPEAKER_06] It just has a script in it and the script is a string. [SPEAKER_06] And inside the string is a reducer and an after append hook. [SPEAKER_06] And so this is literally a stream processor. [SPEAKER_06] It would be funny if this is the thing that works because I was basically up most of the night working on this. [SPEAKER_06] Because this is the most exciting bit, but it wasn't really workshop grade because you get the first five concepts. [SPEAKER_06] And then you're like, by the way, you can deploy this by just adding an event. [SPEAKER_06] But if I just say ping here, it says pong, right? [SPEAKER_06] That's pretty crazy.
SPEAKER_06
[SPEAKER_06] I have here a stream that knows nothing about the world. [SPEAKER_06] And then I have appended this event here that just has a little bit of JavaScript in it. [SPEAKER_06] And this little bit of JavaScript just makes it respond with pong whenever it encounters something with ping. [SPEAKER_06] And then there is actually a version of this. [SPEAKER_06] This is the thing that actually took the night away, so to speak, is if you have these processor files that export a processor that you define with defined processor, you can bundle that into an event effectively. And this AI agent, I also got running like that.
SPEAKER_04
[SPEAKER_06] But of course you don't want to have your open AI API key in there. [SPEAKER_06] So that is why we have this slightly awkward env vars here where you can put your open AI API key outside of the stream. [SPEAKER_06] And then when it makes a fetch request, it will substitute the secret in the header.
SPEAKER_03
[SPEAKER_06] But yeah, I'll definitely post a video of that because that's pretty wild.
SPEAKER_03
[SPEAKER_06] You can write the 40 lines of code required for a basic AI agent, and then you can append that to any stream.
SPEAKER_06
And then that stream becomes an AI agent.
I don't know. I'm sort of looking at two or three nods, but mostly blank stares. It is very strange. But I think does this make sense what happened here at least? Because you can, this one you can just play with, you can just go in the UI. [SPEAKER_06] And so to some extent, you can make increasingly more complicated processors for your streams. [SPEAKER_06] You can just code them in this input field if you want. [SPEAKER_06] Or you know, the only problem is it cannot have any dependencies.
SPEAKER_04
[SPEAKER_06] If you went to NPM packages, you have to bundle it into the string, which gets a bit more complicated. [SPEAKER_06] But yeah.
SPEAKER_04
[SPEAKER_06] Can you go overwrite? [SPEAKER_06] But I think does this make sense what happened here at least? [SPEAKER_06] Because you can, this one you can just play with, you can just go in the UI.
SPEAKER_06
And so to some extent, you can make increasingly more complicated processors for your streams. You can just code them in this input field if you want. Or the only problem is it cannot have any dependencies. If you went to NPM packages, you have to bundle it into the string, which gets a bit more complicated. But yeah. Can you go overwrite? Yeah, so this is the other thing. Everything that you might have noticed is everything has a slug.
SPEAKER_05
[SPEAKER_06] It's just a URL safe identifier. [SPEAKER_06] And so the event here is actually called dynamic worker configured because you can override it. [SPEAKER_06] And what also already works with this event type is you can give instructions to an AI agent to make itself different functionality just by calling append. [SPEAKER_06] With a little bit of a different JavaScript. [SPEAKER_06] And you can quite easily imagine overcoming the bundling problem, for example, by just saying, oh, there's another event type, which is called unbundled dynamic worker, which just has a package dot JSON file, a field and a script field.
SPEAKER_05
[SPEAKER_06] And then a different processor takes it, bundles it, makes the fat event that has all of the bundled source code. [SPEAKER_06] And suddenly it's basically an IDE. [SPEAKER_06] And that's then where you can hack on the agent harness without having to install anything or bring servers into the mix or whatever.
SPEAKER_06
But sadly I was not able to show it. So bummer. What if there is an error in the JavaScript code you put in the screen? Hmm? So if you put JavaScript code in the screen and if there is an error in that code or if you made a typing error, what happens there? Well, this is JavaScript. There wouldn't be a typing error. But if there's any kind of error, the only thing that can possibly happen is that you get an event. Yeah. So there would have to be an error event. And there's no typing errors as implemented right now, but that's because this is a JavaScript evaluator.
SPEAKER_06
But you could also find a way to run a TypeScript compilation step on it first and then you could emit an error saying there was a TypeScript compiler error. Then you could run it through an LLM to try and fix it and emit a new event with fixed code or do something else with it. [SPEAKER_00] Yeah. [SPEAKER_00] The idea is that just everything that happens results in an event and then you can react to those events however you see fit. [SPEAKER_00] Yeah. And the events are then evaluated and deployed your second, right? Yes. So in this particular case, it's dynamic workers in Cloudflare. So it just spins up a small dynamic worker.
SPEAKER_06
Basically the deployment story for a processor like this is, or one way to think about it is, when this was still working before I asked it to be refactored and broken, I had the way I would do it is I would locally on my computer do the SSE subscription and run my processor that way. [SPEAKER_04] And then once I thought it was cool, I could either deploy it just as a web service on Cloudflare workers or Vercel or wherever, and tell the agents to notify me whenever a new event needs to be processed or alternatively just use this dynamic worker thing. [SPEAKER_04] And there's, I don't know if this is going to work.
SPEAKER_06
[SPEAKER_07] This is why I like at the moment it's looking more like it's a really dumb idea, I think, because it's really not coming across. [SPEAKER_04] But for example, you could have a plugin for your agent that runs on another computer, especially if it was something really important, like a prompt injection protection or something like that. [SPEAKER_04] You could say, okay. [SPEAKER_01] The way that my agent uses your prompt injection protection service, it can't really be through MCP because there's no way to proactively hook into the agent loop, or at least not at the minute.
SPEAKER_06
But what it could be is it could say, okay, my agent loop actually waits for up to 200 milliseconds before making any new LLM request for any safety checkers to say, and then you can just ask your safety checker over there. I don't care how you implement it. You could even charge me for it. Right.
SPEAKER_04
[SPEAKER_06] And I just don't think that's currently possible in any other way.
SPEAKER_06
That's why I think it's quite important that the stream can reach out to the processors and say, you gotta do something, you need to process some events now. There's a couple other, potentially, I don't, I mean, this might.
SPEAKER_04
[SPEAKER_06] Yeah.
SPEAKER_06
Yeah. Well, I mean probably initially the human would be charged for it, but this is just, there is no way. Let's say okay, so cloud code. If I want to make a cloud code plugin, if I wanted to make that into a business, think about how you would have to do that at the moment. And you need to give some sort of instructions, or something to cloud code either to proactively call your MCP tools or call maybe some CLI tool that was previously authenticated where you did CLI log into, you know, prompt injection protector.com. And then somehow it works, whereas really, I think you can just plug into the agent stream, into the event stream directly.
SPEAKER_06
The interface is just somebody notices that cloud is about to do something and chuck some more events in and they kind of all work that way, but not quite. It's you have to squint a lot to see that that's what it all compiles down to. Another thing may be worth pointing out for some of you that have tried to make open code or PI plugins or whatever is there's no before hook. This is very, very important. There is actually in the backend, our built in processors. The only difference to our processors and your processors is that we can actually stop events from ever being appended.
SPEAKER_06
[SPEAKER_01] For example, that is why the pause feature could not be implemented by a third party. So there's certain things that just need to happen before an event goes in the stream. But broadly speaking, I'm very against before hooks. Another thing worth pointing out for some of you that have tried to make open code or PI plugins or whatever is there's no before hook. This is very, very important. There is actually in the backend in our built-in processors. The only difference to our processors and your processors is that we can actually stop events from ever being appended.
SPEAKER_06
[SPEAKER_01] For example, that is why the pause feature could not be implemented by a third party. So there are certain things that need to happen before an event goes in the stream. But broadly speaking, I'm very against before hooks. I think there were some instances in open claw, for example, some massive performance regressions and cost increases where you can break context caching quite easily with before hooks. You can massively destroy performance.
SPEAKER_06
I guess it's way better to think of the whole system as being eventually consistent and doing all these distributed systems things like saying, okay, we're going to wait for up to 200 milliseconds for somebody to say, I've got a little bit of information for you, but then we're actually going to make the LLM request whether you come in with information or not, because this doesn't need to be a really brilliant system.
SPEAKER_06
So if you think, for example, about RAG pipelines and things like that, you can easily have a little processor that says, I have an indexed version of the Notion knowledge base or something, and I'm just going to sit here and try to squeeze in a little bit of extra context if I think it's relevant. But if I don't get there in time, it's totally chill. The whole thing still works. It's not like I've delayed the agent or something. So you can do that. Do you have a question? How would you imagine in the future would everyone have their own instance of this or would they have a specialized one?
SPEAKER_06
The streaming database, I don't know, I think this could work on almost anything that is like a durable stream shape. This was just for the purposes of this exercise. I do think there is potentially an interesting scenario where it's a combination of a queue and a pub sub system and a streaming database and that can also run code for you. But that sort of thing is an interesting infrastructure primitive that maybe somebody could make. In principle, if you don't know about it yet, you can read up on the durable streams API specification that somebody came up with. We don't follow it exactly, but close enough. And it's basically a very common old idea.
SPEAKER_06
[SPEAKER_03] The thing that is more new is that it would have pushed subscriptions out of it. [SPEAKER_03] But there are some things that do this. There's a thing called Nats, Jetstream Nats or something, that does it. Are you talking about Electric SQL? Yeah, exactly. They have durable streams, yeah. And a durable stream is just an append-only event log with offset tracking. The client tracks the last consumed offset. [SPEAKER_06] That's also how Kafka works or something like that. But you can also do it the other way around and have the server track the offsets.
SPEAKER_06
Those are more normally like this. Google Cloud and AWS, they have these pub sub systems where they say, oh, we'll just give you the next event and we'll keep track of the offset for you. [SPEAKER_05] Say I've got Michael Bell here on my and I have a sub particle images and I hook it up to a local LLM which is generating images. And then anyone could essentially pass the stream to that, which would then go to my LLM generated image and return it back. How would I prevent a certain amount of, maybe good flooding? Oh, I mean, you would need to have authentication on it. Right. This entire thing would never work like this.
SPEAKER_06
I was thinking it might be fun to just say there's a, there's like. [SPEAKER_03] Yeah, exactly. [SPEAKER_03] It might be fun to just say there's a public namespace and it just gets cleared every hour or something. [SPEAKER_03] Right. [SPEAKER_03] I don't know if I've done it because it is kind of nice that anybody anywhere on the internet can just curl a thing or copy and paste a curl from somewhere and suddenly have a flavor of agent.
SPEAKER_00
[SPEAKER_03] I do think that's quite interesting, but it just needs to be deleted every quite aggressively. [SPEAKER_06] Yeah, but I think that's more or less a solved problem. [SPEAKER_06] It's a bit gnarly, but basically the way I envision it, you would do it if you did this properly is on each event you basically have client provenance information.
SPEAKER_06
We say like, who was this? What were they able to do? How sure are we that these people, the HTTP client or the stream client was actually entitled to do this operation? [SPEAKER_06] And again, you can model all of that as events. You would just have an event that is make this stream public.
SPEAKER_04
[SPEAKER_06] And until you make that event, it can only be written to by whoever created it or something. [SPEAKER_06] I might ask Sean if we can, instead of this video, do a video we'll record a little bit later today that shows it more nicely because I do think it's worth showing. [SPEAKER_06] Yeah.
SPEAKER_07
[SPEAKER_06] Just a little bit too short notice.
SPEAKER_04
[SPEAKER_06] Shall we just wrap it then if I don't know, I don't mean to keep you all for ten minutes longer break. [SPEAKER_06] Or if anybody wants to chat.
SPEAKER_01
[SPEAKER_06] Thank you.
SPEAKER_06
Thank you. Thank you. How are you? How are you? How are you? How are you? [SPEAKER_07] How are you?
SPEAKER_06
How are you? How are you? How are you? How are you? And it, and you need to give some sort of instructions, uh, or, or something to, to, uh, to cloud code either to proactively call your MCP tools or, or something of that nature or call maybe some CLI tool that was previously authenticated where you did like CLI log into, you know, like prompt injection protector.com. And then somehow it works where, whereas, um, uh, really, I think you can just plug into the, into the agent stream, like into the event stream directly.
SPEAKER_06
Like the, the interface is just somebody, somebody notices that cloud is about to do something and chuck some more events in and, and like, they kind of all work that way, but not quite. Like it's, uh, it's sort of like you, you have to squint a lot to, to see that that's what it all compiles down to. Um, um, another thing may be worth pointing out for some of you that have tried to, like, try to make open code or, or, or, or PI plugins or whatever is there's no before hook. This is very, very important.
SPEAKER_06
There's no like, um, there is actually in the backend, like our built in processors, the only difference to our processors and your processors is that we can actually stop events from ever being appended.
SPEAKER_01
For example, that is why the pause feature could not be implemented by a third party.
SPEAKER_06
So there's sort of like certain things that are just, um, that, that need to happen before an event goes in the stream. But, um, broadly speaking, I'm very against before hooks. Like I, I think there were some instances in, in open claw, for example, some massive performance regressions and, and cost increases where, you know, you can, you can break, um, you can break context caching quite easily with before hooks. You can massively destroy performance.
SPEAKER_06
I guess it's way better to think of the whole system as, as being eventually consistent and, uh, you know, um, doing all these distributed systems things like saying, okay, we're going to wait for up to 200 milliseconds for somebody to say, I've got a little bit of information for you, but then we're actually going to make the LLM request whether you come in with information or not, because this doesn't need to be a really brilliant system.
SPEAKER_06
So if you think, for example, about rag pipelines and, and, and things like that, you can easily have a little processor that says, I have like a indexed version of the notion knowledge base or something, and I'm just going to sit here and try to squeeze in a little bit of extra context if I think it's relevant. But if I don't get there in time, it's like totally chill. The whole thing still works. It's not like I've, I've kind of delayed the, um, the agent or something. Um, yeah. What about, um, so you can, um, do you have a question?
SPEAKER_06
Do you obviously think this, this instance of x2x, like how would you imagine in the future would everyone have their own one or would they would have a specialised? I mean, the, the, the, the streaming database, like, I don't know, I, I think, um, I think this could work on almost anything that is like durable stream shape. So this was just for the purposes of this, um, exercise. Uh, I do think, like, there is a potentially an interesting scenario where, uh, like, where, like, it's not even really like, it's a combination of a queue and a pub subsystem and, and like a streaming database and, uh, uh, that can also run code for you.
SPEAKER_06
But, but that, that sort of thing is an interesting infrastructure primitive that, uh, maybe somebody could make. Um, I do think that, uh, yeah. But in principle, like, I, I don't know, if you don't know about it yet, you can read up on this, uh, durable streams, uh, API specification that somebody came up with. We don't follow it exactly, but, uh, close enough. And it's, it's basically like, like a very common old idea.
SPEAKER_03
Like, um, the, the thing that is more new is that it would be, like, you would also have pushed subscriptions out of it. But there are some things that do is there's a thing called Nat, uh, like Jetstream Nat or something, uh, that, that does it.
SPEAKER_06
Um, yeah. Are you talking about, uh, like the electric sequel? Yeah, exactly. They, they have durable streams, yeah. And, uh, like a durable stream is just like an append only event log with, with offset tracking. And, and the, the, the, you know, they want the client to track the last consumed offset. That's also how, say, Kafka works or something like that. Uh, but you can also do it the other way around and have the server track the offsets. And those are more, more normally like this. Google Cloud and AWS, they have these pub subsystems where they say, oh, we'll just, like, give you the next event and we'll keep track of, of the offset for you. Um. Yeah.
SPEAKER_05
Say I've got a, I've got Michael Bell here on my, and I have a subparticle images and I hook it up to like a local LLM which is generating images.
SPEAKER_06
And then anyone could essentially pass the screen to that, which would then go to my LLM generated image and return it back. Yeah. How would I, like, prevent a certain amount of, like, maybe good flooding? Oh, I mean, you would need to have authentication on it. Right. Right. Like, this, this entire thing would never, like, right, like this. Uh, I was thinking, like, it might be fun to just say there's a, there's like. Don't do that yet.
SPEAKER_03
Yeah, exactly. It might be fun to just say there's a public namespace and it just gets cleared every hour or something. Right. I don't know if I've done it because it is kind of nice that anybody anywhere on the internet can just curl a thing or like copy and paste a curl from somewhere and suddenly have like a flavor of agent. I do think that's quite interesting, but it just needs to be deleted every, like, quite aggressively.
SPEAKER_06
Yeah, but, but I think, I think that's more or less a solved problem. It's like a bit gnarly, but basically the way I envision it, you would do it if you did this properly is on each event you basically have like client provenance information. We say like, who was this? Like, what were they able to do? Like, how sure are we that these people, you know, the HTTP client or the, the stream client was actually entitled to do this operation. And again, you can model all of that as events, right? Like you can, you would just have an event that is like make this stream public. And until you make that event, it can only be written to by whoever created it or something.
SPEAKER_06
Right. I might ask Sean if we can, instead of this video, do a video we'll record a little bit later today. Um, that shows it more nicely because I do think it's worth showing. Yeah. Just a little bit too short, short notice.
SPEAKER_06
Shall we just wrap it then if, uh, I don't know, I don't mean to keep you all of your, uh, like, ten minute, longer break. Or if anybody wants to chat. Thank you. Thank you. Thank you.
SPEAKER_06
How are you.
SPEAKER_06
How are you. How are you. How are you.
SPEAKER_07
How are you. How are you. How are you. How are you. How are you.