You Didn't Ship a Bug. You Just Wrote It for a Human. - Ravi Madabhushi, Scalekit

Transcript

Hi, thank you so much for tuning in. I'm Ravi. I'm one of the co-founders of ScaleKit. Today, I'm going to talk about how you need to think architecturally from the ground up about building your applications, APIs, your MCP servers for agents, and how the human-focused architecture doesn't scale well for agents. A while back, we were looking at our performance and latency numbers. One thing that jumped out at us was how our latency was spiking every 15 minutes in a rhythmic manner. Nothing harmful, but just a curious thing for us to analyze. What we noticed was very interesting. In our identity and authentication infrastructure platform, we have this little timestamp that we mark for every user to say, hey, when was the user last seen, or when was the user last active or last acted in our system, so that we can predictively say, hey, this user is an active user. This user is not so active. But one thing that we realized was this system was probably built for humans. When agents started hitting our APIs in the last 12 months or so, we realized that this last seen update is happening 60 times faster than what it would. That is creating unnecessary pressure in our dbWrite system. Of course, it's a harmless thing. We were able to fix it very easily. We could just batch the update at a second level and not every single time we had to update it. That took us down a rabbit hole. The assumption that broke was how often our system would have to update this timestamp on every row. That's okay. It's just about speed. It's about latency, etc. But what I was worried about is, hey, what if some of our assumptions that we made about authentication and authorization need to be rewired and rethought completely when it comes to agents, because we would have designed earlier for humans as actors in mind? Just to give you context, I worked on identity and authentication operation for the last 10 years, building an identity platform at Freshworks, which is being used by millions of daily users, hundreds and thousands of customers all over the world. But this is predominantly human users, right? Or, the way I think about APIs, APIs are also accessed by machines that are written by humans. That's not too bad, right? But what I realized is the fundamental picture has changed drastically in the last three, four years or so. We have a unique ringside view to see how developers nowadays are building agents and how they're giving context to these agents with data from third-party applications like Salesforce or Databricks or HubSpot or Notion. What we have realized is most of the agents our customers are building have way too many permissions and scopes than the agent's responsibility or the agent's job is. Again, it's not because the developers who are building the agents are careless, but somehow this became a default pattern of giving the agents what they need access to, and the existing primitives that we have don't let us give extremely fine-grained permissions to the agents. Now, I'll tell you how we ended up here, right? We predominantly have two slots, and neither of the slots was built for agents in mind. There's a human who's accessing the application, either a web application or a mobile application or their own little script that they wrote, and they give it their API key so that their program can access data from the application. The fundamental principle here is it's the same user who is authenticating, and it's the same user who is acting, right? The second slot is the traditional service account scenario or m2m account scenario, where you create a service account, you give it certain permissions, and then say this machine has its own identity. That's where the likes of SPIF and OAuth and all of that came into the picture, but you would give them certain credentials and say, hey, now this machine has access to whatever data that it needs at any single point of time. This is the existing pattern, right? The fundamental philosophy that we have always maintained is whoever is authenticating is the one that is acting. Every action the program or the human takes is based on a fixed set of permissions that actor was granted at some time. If you take traditional authentication mechanisms for humans, including password, you just say, hey, if an identity has the same password that it was set at the time of registration, if they come back and if they present the same password again, then you say, okay, this is how I validate the identity. This is how I authenticate the human, and every action subsequently is tied to that human identity. Again, the same is the case with API key, or the same is the case with web session tokens, or even the same case for service account. You define the permissions at the time of registration, and then every single time it acts based on the registration-time permissions and scopes. This is okay all this while because, for decades, the service account and OAuth principle even is working fine, even though there are their own problems. But it is still working fine because these machines are using a program in a deterministic way, by the way the human developer wrote that program. So there are absolute guarantees about what the program could or the program won't do, but it is still intentional based on what the human wrote, right? In this particular case, again, if it is using API keys, then the actor and the principal is the same. Then there is some sort of delegated permission for the program to act based on what consent the user has granted. But the second one is the most important part, which is it's a deterministic program, and it always stays in its own lane. It can never do what it was not programmed to do, and you could inspect the code to say, okay, is the program doing what it is supposed to do? Even if you apply for a Google developer account and then ask for client ID and you need to access sensitive scopes, you have to go through a security review. What they're doing is looking at your code base to say, are you doing enough checks? Are the appropriate practices in place? So these programs are deterministic. These programs behave the exact same way a developer-programmed program can work. But agents fundamentally break this assumption, right? First of all, in the case of agents, the principal is not the same as an actor. You need to give delegated access so that the agent can act on behalf of the user. Agent can access the user's Gmail. Agent can access the user's Salesforce data. Whatever the case may be. But again, unfortunately, not a lot of systems even today support OAuth. So here again, there is no on-behalf-of principle that is working. So you don't even know if there is a program that is acting on behalf of the user or the user acting by themselves, right? That's a fundamental problem. The second problem is even more dangerous, which is right now the program is not written by human. There is no determinism baked in to say what the agent will do or won't do. Just because an agent does certain things today, you can't be 100% certain that the agent can do the same or the exact same thing tomorrow, the day after, or even in its next immediate run, right? Because of this non-deterministic nature of the agent, we usually pick one of the two lanes, right? You give a specific identity to the agent, which is what we call as client ready in the context of OAuth, and then say, you act on behalf of this particular user, or we go back to the agent acts as the user, which is even worse. The fundamental reason why I'm harping on the same thing is because when an agent is acting on behalf of user one versus user two or user three, the agent needs specific permissions based on the user's context. Now, OAuth solved this perfectly fine by saying, hey, the user will grant specific scopes or permissions to the agent. So the agent is acting on behalf of the user. It can't do everything, but the agent will only do certain things. Now, in the kind of world that we're looking in, most of the MCP servers that we work with don't actually limit the tool context access to the agent based on which user authorize the agent. They typically surface all the tools that the user has access to, or all the tools that the application can even support, and then let the agent determine what they can or cannot do. Now the agent ends up picking the wrong tool, doing the wrong things. Maybe there is some runtime check in the application that prevents some of these things, but the agent is still seeing the same surface regardless of whom it is acting for. Two things that we need to solve for: one is the actor, in this case an agent, has to be bound to the principal at all times. And the agent should have its own identity. Agent should have extremely fine-grained credentials, not the OAuth scopes that we are seeing today. If you inspect the scopes for some of these applications, even very popular applications like Gmail, it will say, can this client send emails on your behalf? There is no extremely fine-grained scoping to say, can this agent act at this hour? Can this agent read emails only from these senders? Can this agent send emails to only this recipient? The reason why that is important is because, again, we spoke about this earlier. In the context of non-deterministic agent workflows, it's extremely important that the agent should have permissions for a limited amount of time that they are operating in. Number one, every agent has a goal. Every agent has a job. So you should be able to deterministically say that this agent should have access only to those tools or only to those jobs it has access to. Gone are the days when the broad-scoped OAuth scopes that we defined were okay because, in that case, the developer was writing a deterministic application. You can review the code to make sure that he's not doing anything sinister. But in the case of agents, it's extremely non-deterministic. It is probabilistic. Agents are bound to do things with whatever they can get a hold of. So, in the context of when you're giving access to the agents, you should be in a position to give extremely fine-grained scopes. It should be at an attribute-level scoping. It should be context-level scoping. It should be principal-level scoping. So all of that is extremely important. Again, I think everyone agrees that agents should be least privileged by default, and they should be able to ask for just-in-time authorization if they want elevated scopes. Now, the reason why we're talking about this is because it's not some futuristic thing. It is happening today. We have seen enough incidents where agents end up doing rogue things. They end up deleting production databases and stuff like that. So how do you put deterministic guardrails in place is an important problem to be solved right now. Again, one of our customers, Ref.tools, they don't even have humans as actors. Their predominant product is about how to give context to coding agents so they can do their job effectively. So they built the entire OAuth scoping, how do you do things the right way, and things like that. So the reason why I give this example is not to say that this is a warning shot, but this is a problem of today and not for tomorrow. Before I go, you have to have absolute visibility into what your agent can do. Every action that's taken in your system: who took it, on behalf of whom, and who authorized it, when was the authorization given, what authorization was given, how long is it given for. If you don't have visibility into all these actions at every single time, and if you can't deterministically control what your agent can or cannot do, then you're just praying that the agent doesn't end up doing what it's not supposed to do. And praying is not a strategy, as we all know. One last thing to take away. If you architected so far with humans and APIs in mind, you need to start rethinking about how you need to give deterministic guardrails and deterministic authorization controls to the agent. OAuth is a good place to start, but you need something beyond OAuth to make sure that the agents have extremely fine-grained access controls and agents are always acting by themselves on behalf of certain users. Thank you so much for your time. password, you just say, hey, if an identity has the same password that it was set at the time of registration, if they come back and if they present the same password again, then you say, okay, this is how I validate the identity. This is how I authenticate the human and every action subsequently is tied to that human identity. Again, the same is the case with API key or the same is the case with web session tokens or even the same case for service account. You define the permissions at the time of registration and then every single time it acts based on the registration time permissions and scopes. Now, this is okay all this while because for decades, the service account and OAuth principle even is working fine, even though there are their own problems, but it is still working fine because these machines are using a program in a deterministic way by the way the human developer wrote that program. So there is absolute guarantees about what the program could or the program won't do, but it is still intentional based on what the human wrote, right? In this particular case, again, if it is using API keys, then the actor and the principal is the same. Then there is some sort of a delegated permission for the program to act based on what consent the user has granted. But the second one is the most important part, which is it's a deterministic program and it always stays in its own lane. It can never do what it was not programmed to do and you could inspect the code to say, okay, is the program doing what it is supposed to do? Even if you apply for a Google developer account and then ask for client ID and you need to access sensitive scopes, you have to go through a security review. So what they're doing is looking at your code base to say, are you doing enough checks, the appropriate practices in place. So these programs are deterministic. These programs behave the exact same way a developer program can work. But agents fundamentally break this assumption, right? First of all, in the case of agents, the principle is not the same as an actor. You need to give delegated access so that the agent can act on behalf of the user. Agent can access the user's Gmail. Agent can access the user's Salesforce data. Whatever the case may be. But again, unfortunately, not a lot of systems even today support OAuth. So here again, there is no on behalf of principle that is working. So you don't even know if there is a program that is acting on behalf of the user or the user acting by themselves, right? That's a fundamental problem. The second problem is even more dangerous, which is right now the program is not written by human. There is no determinism baked in to say what the agent will do or won't do. Just because an agent does certain things today, you can't be 100% certain that the agent can't do the same or the exact same thing tomorrow day after or even it's the next immediate run, right? Because of this non-deterministic nature of the agent, we usually pick one of the two lanes, right? You give a specific identity to the agent, which is what we call as client ready in the context of OAuth and then say, you act on behalf of this particular user or we go back to the agent acts as the user, which is even worse. The fundamental reason why I'm harping on the same thing is because when an agent is acting on behalf of user one versus user two or user three, the agent needs specific permissions based on the user's context. Now, OAuth solved this perfectly fine by saying, hey, the user will grant specific scopes or permissions to the agent. So the agent is acting on behalf of the user. It can't do everything, but the agent will only do certain things. Now, in the kind of world that we're looking in, most of the MCP servers that we work with don't actually limit the tool context access to the agent based on which user authorize the agent. They typically surface all the tools that the user has access to or all the tools that the application can even support and then let the agent determine what they can or cannot do. And now the agent ends up picking the wrong tool, doing the wrong things. Maybe there is some runtime check in the application that prevents some of these things, but the agent is still seeing the same surface regardless whom it is acting for. Now, two things that we need to solve for. One is the actor, in this case an agent, has to be bound to the principle at all times. And the agent should have its own identity. Agent should have extremely fine grained credentials, not the OAuth scopes that we are seeing today. If you inspect the scopes for some of these applications like even very popular applications like Gmail, it will say, can this client send emails on your behalf? There is no extremely fine grained scoping to say, can this agent act at this hour? Can this agent read emails only from these senders? Can this agent send emails to only this recipient? The reason why that is important is because, again, we spoke about this earlier. In the context of non-deterministic agent workflows, it's extremely important that the agent should have permissions for a limited amount of time that they are operating in. Number one, every agent has a goal. Every agent has a job. So you should be able to deterministically say that this agent should have access only to those tools or only to those jobs it has access to. So gone are the days when the broad scoped OAuth scopes that we defined is okay because in that case, developer was writing a deterministic application. And you can review the code to make sure that he's not doing anything sinister. But in the case of agents, it's extremely non-deterministic. It is probabilistic. Agents are bound to do things whatever they can get a hold of. So in the context of when you're giving access to the agents, you should be in a position to give extremely fine-grained scopes. It should be at an attribute level scoping. It should be context level scoping. It should be principle level scoping. So all of that is extremely important. Again, I think everyone agrees that agents should be least privileged by default and they should be able to ask for just-in-time authorization if they want elevated scopes. Now, the reason why we're talking about this is because it's not some futuristic thing. It is happening today. We have seen enough incidents where agents end up doing rogue things. They end up deleting production databases and stuff like that. So how do you put deterministic guardrails in place is an important problem to be solved right now. Again, one of our customers, Ref.tools, they don't even have humans as actors. Their predominant product is about how to give context to coding agents so they can do their job effectively. So they built the entire OAuth scoping. How do you do things the right way and things like that. So the reason why I give this example is not to say that this is a warning shot, but this is a problem of today and not for tomorrow. Before I go, you have to have absolute visibility into what your agent can do. Every action that's taken in your system, who took it, on behalf of whom, and who authorized it, when was the authorization given, what authorization was given, how long is it given for. If you don't have visibility into all these actions at every single time and if you can't deterministically control what your agent can or cannot do, then you're just praying that the agent doesn't end up doing what it's not supposed to do. And praying is not a strategy as we all know. One last thing to take away. If you architected so far with humans and APIs in mind, you need to start rethinking about how you need to give deterministic guardrails and deterministic authorization controls to the agent. And OAuth is a good place to start, but you need something beyond OAuth to make sure that the agents have extremely fine grain access controls and agents are always acting by themselves on behalf of certain users. Thank you so much for your time.