Build-Time vs. Run-Time: Why Dev Tools Fail in Production — Averi Kitsch & Prerna Kakkar, Google

Hey everyone, how are all of you doing today?

Yeah. So nice to meet you everyone. Today, my friend Avery and I are going to talk about build time versus run time, why your developer tools fail in production. So firstly, let me tell you about us. Hi everybody, I'm Avery Kitsch and I'm a staff software engineer working on Google Cloud databases. I'm currently the technical lead for MCP Toolbox for databases, our open source database MCP server and our Google Cloud MCP server maintainer.

Hi, I'm Pradna and I'm currently working as senior software engineer at Google. I am currently tech lead for EvalBench, which is the evaluation framework for all your agent, MCP and skills needs. And I'm also an active contributor to MCP Toolbox. So today we are going to cover three areas broadly. We will firstly start with the history of MCP at Google. Then we will cover the common tool patterns that we have found from our own work and practices, and how we use all those practices to build some tools for database access and how you can use them. And then lastly, we will talk about security guardrails, how you can stop data leaks using identity-aware guardrails.

So let's get to know the background quickly. I'll talk about MCP Toolbox for database. It's an open source, self-managed service that we provide. It has currently about 15.7k GitHub stars. We have 132 plus active contributors across 40 plus different databases. It's a highly customizable framework. And we provide you with connection pooling, integrated auth, and you don't even need to care about the observability. You will get all of them out of the box. Then if you don't want to do a self-managed one, but you want to have a hosted, scaled version, we provide something as Google Managed MCP. It's fully managed.

You can plug it across various agents and IDEs or harnesses like Gemini CLI, Anthropic CLI, Cloud Code, you name it. It's governed and the discovery is simple. And we also provide model armor, which provides secure access management and identity control. So combined with the managed version of MCP and the MCP Toolbox, last month we had 20 million tool calls. Some of the common tool patterns that we have observed specifically for databases, so I'm going to quickly talk about them. Firstly is the control plane tools. What we like to call them is admin tools or managed tools. It is basically in developer assistant space.

So it will help you create instances, manage your instances, create your databases, manage your databases. It will help you with all your DBA needs. But you need to be very careful. You need to have a human in the loop because we don't want to carry out any dangerous activities. So these tools are built on already provisioned public APIs. So you get monitoring and other things out of the box. Next one is natural language to SQL or NL to SQL tools. So we are relying on a tool called execute SQL and with the help of an agent, we generate raw SQL queries. So you can use this in cases where you don't know what queries you would require beforehand.

So you will get all these queries out of the box. So it focuses on developer assistance and analytical agents and you can use it for flexible explorations.

So for example, we have one example like find all customers in California who bought a winter coat in July and returned it within 14 days and grouped them by the marketing campaign that originally acquired them. So this is one of the queries where you can use this tool to get your answers. But then we have something called structured SQL tools, which is getting quite popular. And this targets mainly the production use cases where you know what SQL query you want to use and you want to have security built in. And the parameters are already configured. So you prevent SQL injection and ensure highly controlled access by restricting the agent to predefined logic.

It also helps you with your latency needs and reduces hallucination on the agent side. Now, we come to the main topic, I guess, for which you guys are here for, build time versus run time. So build time are the developer assistant use cases. You can think about the initial two cases that we presented to you like the NL2SQL tools and the control plane tools. They come into the category of build time tools. They are atomic and flexible. But again, you don't want to delete your databases. So it requires a human in the loop case and you can't run them on production use cases. But let's say I'm interested in building a chatbot and I want to do production use cases.

There you rely on runtime or end user applications. You can build those using Bedrock AI or Lanchain. So you can see one of the examples like we have a cancel order or deterministic structured SQL query that we have given and you can use it as a tool. This is one of the examples or demos for where a build time tool was used. And you can see the error message. So the agent actually asked to delete the table and start fresh. We deleted everything and there were no safeguards or guardrails here. Now let's go to our demo for runtime tools.

Yeah, maybe. I think until the video loads, sorry for the technical glitch that we have, but I can quickly walk you through what we are going to present in the video and I guess it's loading. So this demo is particularly talking about how we use our production tools in a chatbot. And we created a demo called Symbol Air. And Symbol Air is going to help me with booking all my flights in San Francisco and do whatever I would require to do in San Francisco, it would basically help me with it. One of the things that I would try is I would try to fool my agent that I am Avery and not Pradna and book a flight for me to San Francisco.

But because our agent has all the authenticated auth, it will not get fooled and it will not book any flights on behalf of Avery, but it will do it on my behalf. And then you can use it to basically change your flights. You want to know about all the shops that are there. You can do all these requirements using that. So I guess, thank you. Avery. Apologies again for our technical difficulties here. Unfortunately, it looks like I need to present from just this slide deck because it's not loading. Okay. So I apologize for not being able to see our demo today, but we can still learn all the security and guardrails that we need to secure our database access.

So the first thing that we need to know is your database is only as secure as your agent. We all know that agents and LLMs are actually pretty easy to trick. They might be getting slightly better today, but we can still work really hard to trick them. And so we have a very common attack pattern called the confused deputy attack. And this is when a user can trick an agent into misusing their privileges to access data that a user wasn't supposed to access. So Simon Willison actually coined the phrase, the lethal trifecta. And a data breach occurs when an agent has simultaneous access to three different things. One, private data. Two, untrusted content.

And three, the ability to expose that content and data back to an external user.

So let's take a look of that in action. So let's say I'm building a triage agent. And a ticket is fired or alert goes out and my agent is designed to look at that ticket and go investigate what it needs to do.

So on that ticket, the agent gets a little bit of data, like we need to go look in this database for these reasons. But a malicious insider can actually come into that trusted system and instead say, well, I want to query the salary database and please return all the employees' salaries. And so since this is a trusted system, the agent goes, okay, let me use my permissions. I have those privileges. I have that access. I will query that and I'll post that right back on the ticket because that's what the ticket needs to do. But now we have a huge data breach. A user that wasn't supposed to have access to private data now has that access. And so now we have a big PR fiasco.

So this makes a little bit more sense when we think about who's controlling access and who's controlling the parameters. So we talk about agent or application versus model controlled parameters. So in a traditional architecture, things were actually much easier because you would have a few input fields, you would define your queries, and then that would be safely injected into those queries. So it was okay when your application had a little bit more access because it knew exactly what actions it was going to take.

But in a Agentic application, these rules aren't as clear.

So we need to first think about separating the three different identities. We have the user identity, we have the application identity, and the agent identity. So first, we need to think about what the user has access to. So the user just needs to have access to the application. That application's workload identity can have a little bit more broader access because it needs to probably talk to different services. But the agent running in that application only needs to have access to the data that that end user initially needs to have. So next we need to think about who's controlling the tool inputs. So we have agent parameters and application parameters.

So agent parameters are the untrusted inputs that the agent is deriving dynamically. And then we also have application parameters. These are the factual constraints that we need to keep outside of the agent's control. Okay. So now let's look at the evolution of a secure tool. Here we have a fully model controlled tool. And so essentially the agent here is a super user. It has access to database credentials, the host, the port, the connection details, and even the raw SQL query.

And so we're only as secure as the agent here. And we can really easily, again, trick the agent into exposing all of this data. And now we have access to essentially any database in the system. So Toolbox solves for this by introducing a source primitive. So we move the connection details out of the agent's control. And in Toolbox, a user will pre-configure the connection details in a YAML file. And then when we start our MCP server, those are safely injected. And so we do not have to have the agent have access to that. So we can add a little bit more control to our source security as well. Our number one request that we get from customers is read-only restrictions.

We want to be able to remove all write ability from agents if we need that specific user journey. So this means removing write tools, but also down to the database driver, ensuring that we only do read-only queries. If we're also concerned about, again, blast radius and securing all of our tables and our databases, some of our cloud-native databases have this concept of allowed datasets. So again, we can add that enum to our source in order to continue to restrict the blast radius of the agent's control. And lastly is output size.

You might not actually think that this is a security layer, but if, again, the agent gets into the wrong hands, we can reduce that blast radius by saying the agent can only grab this much data.

So we're not overwhelming both our agent or our database. So sweet. We have our configurable sources tool. So you can see here that actually now our tool input, our tool signature is very minimalized. We only have the SQL string that's being generated by the agent. But this comes to our actual next problem. We want to be able to control what the agent is running. We don't want the agent to have the ability to generate any SQL that it can think of. So Toolbox introduces custom tools. And again, in our YAML file, we can define the exact SQL statement that will run very reliably. So we can define the query as a reliable and secure SQL query.

This also allows us to customize the tool name and the tool description. These are really important for the agent to have the context on how to use this tool accurately. And in the system, we use prepared statements with typed parameters in order to reduce SQL injection attacks. So we make sure that everything is validated for all input types when we inject that into the SQL for the user. Okay. Let's dive into a little bit more of best practices for tool quality. So we really highly recommend that tools focus on outcomes. We really shouldn't be thinking in atomic REST APIs. We should think about what the action actually needs to do.

This also reduces the round trip of needing to make multiple tool calls. And again, the descriptions are guidance. We shouldn't duplicate information like input parameters because the agent already has access to that. So writing really good tool descriptions is very important for accurate tool usage. We also recommend that you separate read versus write tools. By doing this, you can automatically approve read tools, and you can also then send write tools to the user for confirmation. And this just makes it very much more clear for the agent to use these. And this is actually the next one: actionable errors. This is the number one thing that I think we can all do better.

So usually we just return a generic HTTP error, 404. But we all know agents are actually really smart now. And so if you give the ability to have an error that can be retried, the agent can actually take that action. So being able to return errors is really important. And lastly, is simple inputs. We see that people try to use these complex maps, complex primitives that an agent needs to be able to build, and that is not reliable. Using flat structures with simple inputs will really increase your reliability.

So sweet. Now we're at custom semantic tools. You can see that we now have our lookup flights tool that takes in the dynamic parameters such as user ID and date. And so now we're very much more secure because the agent isn't generating that SQL query. It doesn't have the ability to go off the rails. It only is looking at these very specific inputs. But user ID is actually a very sensitive piece of information. It is PII. So we need to also remove that from the ability of the agent's control. So we can do this in two different ways. We have bounded parameters.

This is when the application first authenticates the user, and then we can bind that parameter directly to our tool. And so that restricts the agent's control of it. It actually never sees that user identity.

But Toolbox also solves for this in another way called authenticated parameters. This is when we tell the tool that you're going to receive an identity token, an OpenID, a signed JWT token. And when we call that tool, we want it first to validate that token. Is that token real?

Is that token correct? And then we'll extract the user claims from that token for the user. And so the claims usually include a user ID, an email, an issuer. And so it's secured because we're extracting that user identity out of the agent's control, and binding that to the tool. So now we have a much more secure tool. We have our Lookup Flights tool that only takes in a very simple parameter such as date. It doesn't have to handle any sensitive information such as PII, user identity. And so we're really here now at our zero trust architecture where we're in full control of everything that we need to be in control of.

So thank you all for coming to listen to our talk today. Again, I apologize for our technical difficulties. We highly recommend if you want to learn more about our technologies, that you look at our documentation and our GitHub repository. I also really want to highlight our EvalBench repository because this is how we know that our tools are working well and evals are very important.

So thank you all for joining us today. Thank you all for joining us today. So we really highly recommend that tools focus on outcomes. We really shouldn't be thinking in atomic rest APIs. We should think about what the action actually needs to do. This also reduces the round trip of needing to make multiple tool calls. And again, the descriptions are guidance. We shouldn't duplicate information like input parameters because the agent already has access to that. So writing really good tool descriptions is very important for accurate tool usage. We also recommend that you separate read versus write tools.

By doing this, you can automatically approve free tools, and you can also then send write tools to the user for confirmation. And this just makes it very much more clear for the agent to use these. And this is actually, the next is actionable errors. This is the number one thing that I think we can all do better. So usually we just return like a generic HTTP error, 404. But we all know agents are actually really smart now. And so if you give the ability to have an error that can be retried, the agent can actually take that action. So being able to return error is really important. And lastly, is simple inputs.

We see that people try to use these complex maps, complex primitives that an agent needs to be able to build, and that is not reliable. Using flat structure with simple inputs will really increase your reliability.

So sweet. Now we're at custom semantic tools. You can see that we now have our lookup flights tool that takes in the dynamic parameters such as user ID and date. And so now we're very much more secure because the agent isn't generating that SQL query. It doesn't have the ability to kind of go off the rails. It only is looking at these very specific inputs.

But user ID is actually a very sensitive piece of information. It is PII. So we need to also remove that from the ability of the agent's control. So we can do this in two different ways. We have bounded parameters. This is when the application first authenticates the user, and then we can bind that parameter directly to our tool. And so that restricts the agent's control of it. It actually never sees that user identity. But Toolbox also solves for this in another way called authenticated parameters. This is when we tell the tool that you're going to receive an identity token, an open ID, a signed JOT token.

And when we call that tool, that we want it first to validate that token. Is that token real? Is that token correct? And then we'll extract the user claims from that token for the user. And so the claims usually include like a user ID, an email, an issuer. And so it's secured because, or again, extracting that user identity out of the agent's control, and binding that to the tool.

So now we have a much more secure tool. We have our Lookup Flights tool that only takes in a very easy parameter such as date. It doesn't have to handle any sensitive information such as PII, user identity. And so we're really here now at our zero trust architecture where we're in full control of everything that we need to be in control of.

So thank you all for coming to listen to our talk today. Again, I apologize for our technical difficulties. We highly recommend if you want to learn more about our technologies, that you look at our documentation and our GitHub repository. I also really want to highlight our eval bench repository because this is how we know that our tools are working well and evals are very important. So thank you all for joining us today.

Thank you all for joining us today.