SPEAKER_00
Let's get started in an interest of time, right? So, hi, welcome. Let's talk about building agent interfaces today. So, let me start with a question first. Who in here is already using MCP servers or CLI tools on your agent? Okay, everybody. That is unsurprising, to be honest. Who in here have already built MCP servers and deployed them for effect? Okay, it's approximately half of the people. Well, today I'm going to share four engineering lessons from the Chrome DevTools team on how we build Chrome DevTools for agents and how we deployed it for effect.
SPEAKER_00
Quick context setting. Chrome DevTools for humans is used by millions of web developers on a daily basis to debug web pages. It's directly built into Chrome and developers use it to debug web pages, find errors, audit it, performance profile it, and so on. Right. So, now let's talk a little bit more about Chrome DevTools for agents. So, this is a purpose-built Chrome DevTools, but for agents, how surprising. Yeah, let me briefly show you how it works. So, you can see on the left side, Gemini CLI and the prompt is being entered. And now Gemini CLI has the MCP server being configured and it opens Chrome on the right side.
SPEAKER_00
And then does debugging. I think, yes, this is about performance tracing. So, it does a performance trace, analyzes the trace that comes back, go to the performance inside, acts on it, and then makes the web page faster, validate that it's actually faster afterwards, and it's done. You should be done now. It's nearly done. Sorry, it's a video. Whatever. What I wanted to tell you is this is going to work in any MCP client and agent harness that is MCP capable. Doesn't really matter. That was Gemini CLI. Also works in Cloud Code, Codex, OpenClaw, doesn't matter.
SPEAKER_00
If you want to have more information, go to this QR code, because that QR code is going to bring you to a web page, and that's going to tell you everything about how you can install it configured for your agent harness. Question again. Who in here has already tried it out? Okay, so 10%. Thank you. I love you. I also love everybody else, but I love the others more. So, I was rude. I didn't introduce myself. My name is Michael Hablich. I am the product manager for Chrome developer tools at Google, and I'm also a guest director at the university nearby where I'm living.
SPEAKER_00
I have 20 years of experience in tech, developer, tester, QA engineer, project manager, product manager, program manager, and so on. If you have questions afterwards, please talk with me in the hallway track or connect with me over LinkedIn. The QR code will bring you to my LinkedIn page. Both is fine. Please do that. I would be super interested to talk with you about MCP servers, browser automation tools, and stuff like that. Okay, about now, enough advertisement. Let's move on. We ship Chrome DevTools because we saw that coding agents were flying blind one and a half years ago.
SPEAKER_00
They worked very good with generating code, but they were not able to validate what they actually were doing, right? And that just sucked, but we assumed they are going to be fine if we throw a lot of data at them. Because, they are machines, right? The thing is, we were wrong. So this is the head of a trace file. A trace file has all the data about the performance profile. And this is a file, multiple megabytes of data, and this is 50,000 lines of JSON. And we did throw that against common agent harnesses at that point, one and a half years ago, something like that. And without surprise, this is too much data for an agent, for a model to actually reason about.
SPEAKER_00
And it blew through the context window. And if you have seen Matt's talk about the dump zone, you're moving the agent into the dump zone at that point. So I thought, okay, we built it wrong. That's not going to work. We need to do something else. So in that case, for example, what we did, our performance tracing endpoints, it can also return that for post-processing with other tools. But what it's really doing is, it's returning markdown now, and semantic summaries. Like, this is an example of such a semantic summary, which just gives you information about typical performance metrics, like largest content for paint, IMP, and so on.
SPEAKER_00
I'm not going to bore you about all the performance metrics. And we are going to talk about them anyway, because they're a very good example of how this is working. Well, essentially, we didn't force the agent to read the entire book, the trace, but instead we just pointed it at the right sentence, and this is the semantic summary. And that works quite well. In the end, or in the beginning, agents are a different user class. So that's where it clicked for me, yes, they are a separate user segment. So how do we reason about that? The thing is, agents and humans, they share the intent, they share the goal, right?
SPEAKER_00
In our case, for example, both want to identify errors on a page and want to fix those errors. But they think differently.
SPEAKER_00
They have different cognitive bottlenecks, more or less. For humans, it's a lot about visual complexity. So humans are very, typically very visual creatures. And we need layout, we need color in order to find the signal. And this might not be the best example for an appealing UI, but this is just our console interface where console errors are being surfaced, and we can also use it as a wrapper interface on Chrome DevTools for humans. But if you know where to look, you can use it to identify errors on the page very easily. LMMs, on the other hand, they prefer, unsurprisingly, non-visual interfaces, right? They care about schema clarity, data density.
SPEAKER_00
So on the right side, you see a schema for, I think it's for the performance endpoints that we just discussed earlier. Yeah, and they really like that. It's a textual, non-visual interface, right? Whatever. This is now a long way of saying, designing for agents actually requires a few engineering concerns. I'm going to talk about today. So there's token burn rates, there's error recovery, there's tool discoverability, and there's also trust boundaries. And I'm going to cover them today, very shallow.
SPEAKER_00
Let's talk in the hallway track more. Let's start with token burn rate. Tokens are a cognitive load for agents, right? Very similar to humans remembering clicks, the right clicks on a visual interface. The thing is, most APIs today, they're very often a 14-step checkout flow, as you can see on the left side, for example, for a human. Or the trace that I showed you before. I'm going to talk about today. So there's token burn rates, there's error recovery, there's tool discoverability, and there's also trust boundaries. And I'm going to cover them today, very shallow. Let's talk in the hallway track more. Let's start with token burn rate.
SPEAKER_00
Tokens are a cognitive load for agents, right? Very similar to humans remembering clicks, the right clicks on a visual interface. The thing is, most APIs today, they're very often a 14-step checkout flow, as you can see on the left side, for example, for a human. Or the trace that I showed you before. That sucks because every word that is sent back to the model is metered. It costs money, right? Every word, every output token, input token that it gets costs your money. Every reason that's happening costs you money. And that translates to monetary cost. So how can we reduce that? Well, let me introduce a metric. Of course, let's talk about metrics.
SPEAKER_00
And that is called tokens per successful outcome. What it does, it balances effectiveness and efficiency. Sounds very hypey. And maybe it is. I don't know. What is it about? So effectiveness is about does the agent complete the entire user journey? Is the functional intent actually fulfilled? Yes or no? And then there's efficiency, which unsurprisingly is about token cost, tool calls, duration. In the end, what tokens per successful outcome tell you is the fuel efficiency of your interface, right? And there's a caveat because there's always a caveat. Fuel efficiency is relatively worthless if you can't reach your destination.
SPEAKER_00
So that's why it's called tokens per successful outcome and not tokens per outcome. So make sure that you actually also measure effectiveness, right? And there's one more caveat. And this is, you can't measure that globally. I mean, you can do that, but it's going to be tremendously different between different user journeys and task classes. So don't compare them globally. Compare them within your user journey that they're measuring. And what I mean with that is, for example, in Chrome DevTools, we have the user journey of web scraping, right? So an agent going to a website and extracting information. That's relatively cheap.
SPEAKER_00
But there's also user journeys that are more intricate, like debugging a website, finding out why the responsive layout is not working.
SPEAKER_00
That thing is going to use more tokens, but that is fine because it's a much more intricate and more interactive session that's happening. Okay, how does this look in real life? This is what it looks like in practice for a project, an eternal project that we are working on. And you see a lot of neon bars, which is great. I like neon bars. But I am not going to worry about details. The important thing is the neon bars on the left side, the longer the bar, the more effective a tool, right? The shorter the bar, the less effective a tool for a particular use case. So each of those bars on the left side are about use cases.
SPEAKER_00
Which means the smaller bars are probably the ones that we should be focusing on next, how to improve that, how to improve the tokens per successful outcome there. Yeah. As you might have already guessed, measuring tokens per successful outcome is not straightforward. The thing is, but even an imperfect measurement is better than simply doing gut-driven decisions. And with that, at least you can do data-informed decisions.
SPEAKER_00
Right. In DevTools for Agents, we are addressing token burn from three different angles. First, there's tool categorization. So very straightforward. We hide niche tools. We hide command line parameters. For example, we have tools for Chrome extension debugging. And not everybody is developing Chrome extensions. So why add it to the default context window? There's no point in doing that. Then there's a slim mode. And this one is fun. So what slim mode is doing is pushing tool categorization to its limits. It's only exposing, I think, three different tools. Select page, navigate page, and evaluate script. And this is great for your context window. But there's a trade-off.
SPEAKER_00
I'm going to talk a lot about trade-offs today. There's a trade-off because the less tools you expose, the less tools you also have at disposal for your agent, which means your agent might do extra turns to achieve the same goal. It might not actually have the right tools at its disposal to do something. For example, getting network requests. You can't do that with a valid script. And stuff like that. Yeah, and there's also a command line interface that we are offering. You have seen in the previous talk maybe about a code mode and all that stuff. We also support that. So it is an MCP server, yes.
SPEAKER_00
But there's also a command line interface for the same thing, giving them nearly the same functionality. What it enables you, you can have your agent chain commands together to do post-processing. In this example, I don't think you can see it. The accessibility tree is extracted with a grab command, and then the result, the ID of the control is being piped into a click command. And that is, of course, saving a lot of tokens doing that, because the model doesn't need to process all the tokens. The post-processing is happening on your computer. Right. Efficiency is useless if your agent gets stuck. So that brings us to error recovery.
SPEAKER_00
And, yeah, because every time your agent encounters an error, it's going to cost your tokens because it needs to retry, it needs to understand what is happening and stuff like that. And that just sucks. Now, yeah. Error recovery is a spectrum. And let's talk a little bit about that, what we are doing here. So first, of course, you should add useful error messages. That sounds obvious. For a lot of tools, it isn't. And it was also not obvious for all the tools that we are actually offering. So we also did a few iterations on them to actually make the error messages good.
SPEAKER_00
For example, It's going to cost your tokens because it needs to retry. It needs to understand what is happening. And that just sucks. Error recovery is a spectrum. Let's talk a little bit about that, what we are doing here. So first, of course, you should add useful error messages. That sounds obvious. For a lot of tools, it isn't. And it was also not obvious for all the tools that we are actually offered. So we also did a few iterations on them to actually make the error messages good. Here, for example, an unable to navigate back in a selected page for a particular tool, history entry to navigate was not found. We actually added the last sentence.
SPEAKER_00
And that enabled the agent to self-heal, which is super useful, because then the agent doesn't need a human to actually fix the problem, but the agent can self-fix the problems. Then there's proactive detours. So beneath each of the agents, there's a model, and the model is being trained on certain data. And sometimes there are things where you want to contact the training data. And that's what you can do with proactive detours. In this example, we detour the agent for performance profiling to our start performance trace tool and not to the lightest audit. And there's diagnostic playbooks. So we also offer skills, of course.
SPEAKER_00
And we have a skill that's called troubleshooting. And we see a lot of people have problems setting up the Chrome DevTools MCP server correctly. And that troubleshooting skill is then going to kick in and help the human and the agent to fix the setup issues. Again, enabling self-healing of the agent. And all of this increases the resilience of your product, of the agent harness that you're building. And this is nice and helps you with standing mistakes. And now let's talk about this credibility, which is about actually preventing them. So our initial design had one monolithic tool called debug web page. So we all had one tool, debug web page.
SPEAKER_00
And another agent could send a prompt there and tell it, hey, debug this web page. There is some responsive layout that's not working. And it was neat from an engineering perspective, but it didn't really work. So we decomposed it into 25 different tools. And we thought, problem solved. Of course it wasn't. Because we traded that problem off to another. And that was, agents now had 25 tools at the disposal. How are we going to find out which one to use when? Well, let's talk about that. According to this paper here, 97% of MCP tool descriptions have quality smells. And this matters because the schema is the UI for the agent. So let's make the UI better.
SPEAKER_00
And fixing this is a trade-off. As I said, it's always a trade-off. Because, of course, you can make the descriptions better. And that's going to increase your context window size. So you probably don't want to have that. Or maybe you want. And also, smaller models in particular are not that good with more descriptions because they get biased in using tools they shouldn't be using in the first place. This is a trade-off space. Read the paper.
SPEAKER_00
Super interesting. But there are a few things you actually should be doing. They're relatively uncontroversial. And that is, define purpose. Can you explain what the tool call function is? Is this working? Ah, yes. Here it is. Yes. Okay. Explain the tool's core function. And there's usage guidelines, provide clear activation criteria. And how this looks like in Chrome DevTools for agents, for example. And performance.trace tool. What we have in there as a description is used to find performance, front-end performance issues and core web metrics, LCP, INP, CLS. Why is this relevant? LCP, INP, CLS are web performance metrics.
SPEAKER_00
And an agent is able to make the connection, oh, I'm going to use that tool if I need to improve page load, for example. We have far from finished optimizing that because models and agent harnesses also keep on changing all the time. So it's an endless quest for minimum viable description. But it is what it is. You can supercharge all of that with skills. As I said, we also have skills. And that is great. In particular, if you have more intricate workflows. But, again, there's a trade-off. They are not free lunch. If you pile in too many skills, you're going to shift the problem and run into the same problem again.
SPEAKER_00
Agents are going to call your skills even if you shouldn't be calling them. Your context window size is going to increase and all that stuff. The trade-off is shifting. It's not disappearing. Okay, now we have optimized for cost, recovery, and discovery. Let's now talk a little bit about trust because you don't want to have a backdoor in the system. Chrome DevTools for Agents has a feature called Auto Connect. And it's a feature that lets you as a human, using your coding agent like Cloud Code, share the stream with the agent. Hey, I'm stuck here. Please help me debug that and fix the problem that I'm seeing here. Amazing feature. I really like it.
SPEAKER_00
And users, of course, requested a feature. Hey, why do I need to click allow all the time? I don't want to do that. Please remember my choice. And in a traditional user experience design, that would have been a clear win, right? Because it's just friction that you want to remove. In a world where you are delegating away work to agents and automating away agents, you need to think about trust boundaries. And so that's why we actually designed it so that friction is actually by design because we didn't want to have that. And why? Let's talk about that. There's a blog post from Simon Willison about the lethal trifecta. QR code. You should read it. It's great.
SPEAKER_00
I don't want to do that.
SPEAKER_00
Please remember my choice. And in a traditional user experience design, that would have been a clear win, right? Because it's just friction that you want to remove. In a world where you are delegating away work to agents and automating away agents, you need to think about trust boundaries. And so that's why we actually designed it with so that friction is actually by design because we didn't want to have that. And why? Let's talk about that. There's a blog post from Simon Willison about the lethal trifecta. QR code. You should read it. It's great. I'm not going to talk more about that. And utilizing that.
SPEAKER_00
There's at least three tiers that I'm thinking about in browsing agents. You have tier one, and that's the local development environment. In a local development environment, you have the human in a loop. And the human wants to grant access to the default Chrome profile, to the data that you already have access to, to the agent in a time bound manner. And then you have tier two. And tier two is agents running in a continuous integration environment. So it's controlled environments, but they're separated away. At that point, you should be using data separation things like containers, of course, but also other things like separate Chrome profiles and stuff like that.
SPEAKER_00
If you want to connect to them, we also have a mechanism for that, and that is called a remote debugging port. And first, there's agents with full internet access, and that is YOLO mode, essentially, because every webpage out there is able to do some prompt injection text to your agent. So make sure that it's do the same thing as in tier two, but also in tier three, make sure that if the domain allow lists and prompt injection mitigations, all that stuff together. Going back to the lethal trifecta, that's what we mostly reason about tier one, and that's where all those three things are coming together.
SPEAKER_00
So that's why we actually say no, the human actually needs to consent every time. Key point being is a local agent, tier one, and the browsing agent fleet, tier three, your research agents maybe, might share a tool like Chrome DevTools for Agents, but they should share nothing else about your security model that you're having, right? Okay, let me wrap that up. User experience is evolving to incorporate agent experience. An agent is just another type of user, segment of user, also with non-functional requirements. Efficiency, discoverability, security, stability, and so on. I shared four takeaways from Chrome DevTools for Agents.
SPEAKER_00
When we are implementing that, that is measure fuel efficiency of the interface with tokens for successful outcome, turn errors into recovery playbooks, outer descriptions for intent, and never compromise trust for convenience. Agents are our next users. Let's help them help us. And with that, I wish you a nice remaining conference. Thank you. Thank you. Thank you. Agents are our next users. Let's help them help us. And with that, I wish you a nice remaining conference. Thank you.
SPEAKER_00
Thank you. Thank you.