Cooking with Agents in VS Code — Liam Hampton, Microsoft

Transcript

[SPEAKER_00] Hello, everybody. It's great to see so many of you who are still here at the final, on the final day, right at the end. So I hope you've all had a great conference. Show of hands, who here uses GitHub Copilot? Awesome. Lovely stuff. Who here uses VS Code with GitHub Copilot? Awesome. So I'm going to be talking about both these things today. I'm going to be talking about cooking with agents in VS Code. Now, the gentleman before was speaking about cognitive load of agents, and that is absolutely correct. You see so many different things now with agents, and they're popping up all over the place from the CLI, in the terminals, in chat windows, in other editors, etc. But we still somehow seem to find ourselves in this paradigm where everybody thinks agents can solve the world's problems. You still see developers, and I still speak to folks who think we can do one-shot prompts, they'll create a wonderful application or solve all of their issues in one go. Absolutely not the case. And we end up asking these questions from a business perspective of what's the ROI? What's the productivity boost? Where are we seeing our money? And at the moment, we're seeing this whole expenditure on AI and all of this infrastructure, all of these toolings and services, and we're still yet to really reap the benefits of those services. So when we look at how people are spending and how businesses are looking at AI, we really need to be very careful with how we're utilizing the tools and services. We need to be careful about token spend. We need to be understanding the tools and the flexibility. I read somewhere yesterday on LinkedIn, somebody has released this repo, it's growing massively in popularity, and it's talking about having a pirate for your chat bot to talk back to your AI services and language models, because it reduces the token spend. Now people are coming up with these really intuitive and really fun ways to get around token expenditure and really pull in those benefits very quickly. So what I'm going to be talking about is GitHub Copilot agents. Now this doesn't just apply to GitHub Copilot, this also applies to other AI agents as well. So when we're looking at Copilot agents around context, what they really have access to in your workspaces, how they're being used and utilized from within VS Code and the CLI, we're going to be looking at all of those things very shortly. So just a plain and simple, what kind of agents do we have at the moment? Now we're looking at local agents, we've got local agents which are in VS Code, you may use Claude, you may use all these other AI services, still applicable, still running on your local machine with remote models, anything that you're really using, maybe you're using locally hosted ones as well. But this is a way to have local models interacting with you side by side, very hands on, very much in the context and human in the loop. Then you've got background agents. Now we use the GitHub Copilot CLI, we have also got access to that within VS Code, but this is more of an isolated way to be using them. Now we are actually using Git Worktree, show of hands if you know what Git Worktree is and who uses them, awesome, wonderful. For those of you who don't know, an easy way to explain that is it is a branch that is mapped to an isolated folder within the workspace that you're working in, like a sub-directory, just a chop of your code with its own little branch associated to it. Very similar to a Git branch in general. Then you've got Cloud Agents. Now Cloud Agents is quite an interesting one because it allows you to scale outside of your organization very quickly and utilize a lot of the power of the cloud and some of the services that we're using in the cloud. So we use these when we don't want to be touching it ourselves. I use this when it comes to writing documentation or having less of a hands-on approach. So when would we use a local agent? Well, I'd use a local agent when it comes to writing tests. I want to be really hands-on with my tests. I want to understand what's going on in the code base. I really want to be in the weeds. When would I use a background agent? Well, a background agent would be great if I want to be 50-50. I want to create a UI for a front-end of an application. I want to know what's going on. I don't really want to hand it off to a cloud agent because I don't want to be fully out of the loop, but I also don't want to really be hands-on back and forth myself because that can take time. That can be quite arduous. That can be quite annoying. So I would use a background agent. And I'm going to show you how I'm using autopilot to do exactly that with GitHub Copilot in just a moment. When would I use a cloud agent? Well, I would use that mostly for documentation. I hate documentation. I don't like writing it. I don't think many people do unless you're a content developer. I really just pawn that off to the cloud agents. And that could be making a repository open source friendly. It could be writing a readme, using some skills to do that as well. So what I'm really looking at is VS Code as a single entry point for AI agents. We have got third-party support, we have got background, we've got local, and we've got remote entry points for all of these agents. So ultimately what we're trying to do is understand where you are sitting as a developer and how easy we can make it for you to use these agents to reduce that cognitive load. Seems quite complicated but it's actually really straightforward. So I'm going to show a video now. I was going to do this live but I don't really think I'm going to have time to do all of this live. So I'm going to go through this video. So I'm going to start with a very simple Python application. This is just a crud: create, read, update, and delete. Just a very simple product store. Not very pretty. Not very good. As you can see, pretty straightforward. What I actually want to do is create a front-end UI for it. So I've got a ticket up in GitHub and I'm saying, hey, this is wonderful. Going out of front-end. We need some more prettiness here. We need it to look good. So I'm going to say summarize and plan a solution to issue 25. Now you'll notice I'm actually using a CLI background agent at this point. Now I'm using that because I want it to be hands-on, hands-off, a little bit of understanding what it's doing. Also, I don't really care if it messes things up. It can go and iterate. I'm also going to be using autopilot. Now autopilot is currently in preview. And this just means it's not going to ask me a bunch of questions if it wants to do a bunch of tool calls. Great, wonderful, can be very dangerous. Use that carefully, right? Don't just abuse that one. But I'm using it here to create a plan. I don't want it to ask me every single time I want it to do an MTP call. So I'm then saying, wonderful, here's the plan. Now start it, but before you create a pull request, because on autopilot it will do a pull request, stop and pause and let me test locally. Whilst that is off doing its lovely stuff, I can then move on to my next stage where I'm going to be using another kind of agent. So I'm just going to go and leave that one behind. Let's go and spin up a new chat and let's go and start a cloud agent. I've noticed that this is not a very open source friendly repository. I want this to have a readme or have contribution guidelines, have all these readmes that I really want as an open source. But I'm using it here to create a plan. I don't want it to ask me every single time I want it to do an MTP call. So I'm then saying, wonderful, here's the plan. Now start it, but before you create a pull request, because on autopilot it will do a pull request, stop and pause and let me test locally. Whilst that is off doing its lovely stuff, I can then move on to my next stage where I'm going to be using another kind of agent. So I'm just going to go and leave that one behind. Let's go and spin up a new chat and let's go and start a cloud agent. I've noticed that this is not a very open source friendly repository. I want this to have a readme or have contribution guidelines, have all these readmes that I really want as an open source. So I'm going to go and say, hey, go and make this open source friendly, add all the necessary files for it. Now as a developer I can go into my code base and start poking around. I've noticed that I don't have any tests. So I'm going to go check out and I've noticed there is a custom agent available for me in VS Code. This custom agent is essentially explaining and showing how to be using or how to write test cases for this Python application. So what I can do now is start spinning up a local agent. So just like that at the very bottom I can click local. I'm going to select Claudio plus 4.6. I'm going to have medium reasoning. I want it to be fast. It's got a great understanding in this custom agent. Go and write some unit tests. Now as a developer I can still skim through. I've got very much a hands on to and fro with a local agent. I've got a remote agent doing some work for me and I've got a background agent creating a new front end. So here I can see it's written some tests. It's going to go ahead and try and run them. It's passing the tests. But I've also noticed that there's some of the problems in the code. It's not very friendly. The errors that are coming back are not wonderful. So I'm going to say go and update the error handling on the routes and update the tests as well. So you can see I've got a lot of to and fro with this local agent. I've got my remote agent working and I've got my background agent working all simultaneously. So while that's going off and working I can go and check out what my other agents are actually up to. So as I was working through this you can see Copilot is just going to be skimming through. I didn't actually speed up some of this video. This is all pretty quick. I did this pretty quickly. With these agents. There you go. You can see some of the code is updated. We've got the new test. We've got the code updated. Let's go and check out the background agent. That has now finished. Which is cool. Let's go and check on the remote agent. How is the remote agent getting on? Well actually this is the test. Run the test. The tests have passed. That is the local agent. That's now finished. Now I can go and check out my remote agent. So as we're walking through this we can see as a developer I've got very much hands on, hands off. I'm working with multiple agents simultaneously. We can see where they're running all within this single context of VS Code. Now if I go and look on the pull request extension in VS Code we can see that I've now got a pull request. And this is one that I previously run earlier. The one that's running in the chat is actually taking quite a while. But the principle still stands. It's running all these different agents at the same time. So all I really want to do now is go and check out my background agent. I want to go see it working. I want to go see this new front end that I've just created. Now I asked it to pause before I pushed over the pull request and tell me how to test it. So I'm going to say well actually the way you're telling me how to test that is wrong. So I've still got hands on, hands off. It's more of a 50-50. I'm saying this is working in a Git work tree. How do I actually run this? Now remember this is what it currently looks like as an application. So I'm going to go check out the new directory which is a Git work tree. I'm then going to run this Python application. You'll see a very drastic change between what it's created in my single directory versus what it's created. So this is what I currently have. There is a port conflict here. So hurry up and run that. There we are. So likely that this is the new product demo. Now this is the third agent that I'm running simultaneously. And that is essentially a great way of how you're using different agents within one context to kick it all off using GitHub Copilot. That's new error checking. And that is how I've been using multiple agents. So one code base, three problems, three separate agents fixed all at the same time. The local agent was writing my test for me because I wanted hands on. I really wanted human in the loop. I use my background agent to write the front end because I don't really care. What it does is quite an arduous task. It's quite big. It's quite time consuming. And then I use my cloud agent to write my documentation for me. So all in all, that's a pretty successful run. So how are cloud agents actually working? Because I get this question quite a lot. How much is it going to cost? How is it working? What are they doing? And how do I get them running? Well, they're actually running in GitHub Actions. They're pretty safe and secure because they're running in an isolated environment. They have got extended context through MCP servers. Who here uses MCP servers, just out of curiosity? Awesome. So cloud agent actually has access to the GitHub MCP server and the Playwright MCP server. So you can do testing with screenshots. You can do automated front end testing. And you can obviously write your workflows. You've got the dynamic workflows now. And it has got built-in safeguards. So you've got network firewalls. You don't want this agent talking to a whole bunch of different things. It is absolutely whitelisted and restricted. It also doesn't have access to your main branch. Therefore, you're not able to push directly to your main branches. It's very much restricted in that sense. So it is very safe to use. Now I mentioned earlier, this is very much GitHub Copilot. But it is not just GitHub Copilot that this applies to. This actually uses all the same concepts across all different AI agents that you can use. So custom instructions very much defining how the agent is running. You've got custom agents, which is what I showed you today in that short demo, where you're able to use very specific agents to tackle certain problems, i.e. fixing test cases or writing test cases. You have prompt files, which will help you with your prompting and agent skills. And, of course, it is more like the newer version of agents.md, there is always a new thing that is coming on every single week now. So all of this is actually applicable to GitHub Copilot, as well as other AI services, too. Now, inside VS Code, this is a model, which is very recent, and I can jump out of the slides in just a moment to show you exactly how this looks. So custom instructions are very much defining how the agent is running. You've got custom agents, which is what I showed you today in that short demo, where you're able to use very specific agents to tackle certain problems, for example fixing test cases or writing test cases. You have prompt files, which will help you with your prompting and agent skills. And, of course, it is more the newer version of agents.md, and there is always a new thing that is coming on every single week now. So all of this is actually applicable to GitHub Copilot, as well as other AI services, too. Now, inside VS Code, this is a model which is very recent, and I can jump out of the slides in just a moment to show you exactly how this looks. So if I was to go over to VS Code, open up my GitHub Copilot chat pane, and if I click the cog up here, you can actually see everything that I have in one user space for you to customize the chat and the agents that you are running. So whether you've got agents, I've got my custom test agent, I've got my built-in agents, which is Ask, Explore, and Plan. I've got some skills. So this is essentially what some of the VS Code team preempt you to be using here. So you've got some extensions, you've got address of PR, comments, you've got create pull requests. You can jump into these and edit them as you wish. And this is intuitive skills that we have popped in there for you. I don't have any instructions, but this is where you'd have your instructions file, your prompts, if you've got any built-in prompts, like creating an agent, just different prompts that can then go off and kick off its skills. You've got hooks. I don't have any hooks on this one, but a very good example, if you wanted to create or configure some hooks, you can do so with Copilot inside VS Code, and any MCP servers as well. So you have this whole control plane in this modal, which allows you to control your agents and chat customizations from within one single place. This isn't confined. We have third-party support as well. So there is Claude down here, so you can have access to all of your Claude things and all of your plugins, hooks, instructions, and skills for Claude too. So it's not restricted to VS Code and GitHub Copilot. So if you want to get hands-on with some of the skills or customizations, we have got this awesome open source project, which we're running. It's called Awesome Copilot. It's aka.ms forward slash awesome copilot. As I said, this is directed at Copilot, but it's absolutely not just for Copilot. You can use this and massage them and use them for other AI tooling as well, because we do know that people in the community use more than just Microsoft things and GitHub things. We also have an MCP server. So if anybody's interested in utilizing this from their workflow from an MCP standpoint, we also have encapsulated that into an MCP server. For those of you who don't know about MCP, the Model Context Protocol is a great way for you to get hands-on and extend the LLMs that you're working with or any of the chat customizations that you have. For example, if you wanted Azure or to talk to your Azure resources or GCP, AWS, et cetera, you can go through an MCP. It'll obviously be locked down by authentication, but there's also free ones as well and open ones which don't require authentication, like Playwright and documentation ones, for example Microsoft Learn, and so on and so forth. So just in time, as a wrap-up, Visual Studio Code is a single entry point for AI agents, and we're really building this agentic workflow around multiple different services. We've got third-party plugins, we've got first-party plugins, and we've got the full-spec support for MCP. We've got chat customizations, and you can connect to the GitHub Copilot CLI sessions through VS Code. So it's all in one single sequential sequence for you as a developer inside your workflow. I'd love to hear more about your workflows and what you're using and the agents and how you're using them after the session, because I believe I've only got just less than a minute left. So thank you very much for listening, and thank you very much for coming today. applause Thank you. You can use this and massage them and use them for other AI tooling as well, because we do know that people in the community use more than just Microsoft things and GitHub things. We also have an MTP server. So if anybody's interested in utilizing this from their workflow from an MTP standpoint, we also have encapsulated that into an MTP server. For those of you who don't know about MTP, so the Monocontext Protocol is a great way for you to get hands-on and extend the LLMs that you're working with or any of the chat customizations that you have. For example, if you wanted Azure or talk to your Azure resources or GCP, AWS, et cetera, you can go through an MCP. It'll obviously be locked down by authentication, but there's also free ones as well and open ones which don't require authentication, like playwrights and documentation ones, i.e. Microsoft Learn, and so on and so forth. So just in time, as a wrap-up, Visual Studio Code is a single entry point for AI agents, and we're really building this agentic workflow around multiple different services. We've got third-party plugins, we've got first-party plugins, and we've got the full-spec support for MCP. We've got chat customizations, and you can connect to the GitHub Copilot CLI sessions through VS Code. So it's all in one single sequential sequence for you as a developer inside your workflow. I'd love to hear more about your workflows and what you're using and the agents and how you're using them after the session, because I believe I've only got just less than a minute left. So thank you ever so much for listening, and thank you very much for coming today. applause Thank you.