How AI Agents Let GTM Teams Scale — Justin Joyce, Cloudflare

Transcript

Well, thank you everyone for joining. I'm hoping that you guys had a great time so far at this conference, and you guys have a lot of takeaways back to your company. I don't know if any of you guys saw the AGI pills downstairs. Well, I just took some. So if I say any phrases about that's the gun—what's that? What's the phrase? That's the burning gun or the smoking gun, or I start hallucinating in general, please snap me back. That's probably just the AGI pills. All right. So without further ado, let's get started. My name is Justin Joyce. I'm a principal sales operations and strategy manager at Cloudflare. And I work with the go-to-market team as part of the revenue operations organization, specifically on the teams that produce leads for the sales teams, as well as the customer experience team, which works on the customer experience after the sales have been done. And just a little background about me: as Mata said, I started in sales operations and sales. Then I moved to the machine learning side for about the last seven years at Grainger. And I really wanted to do that to be able to learn how to have prescriptive analysis and prescriptive predictions so I could help the business better make decisions and understand what's the next best step. So six months ago, I had an opportunity to move back into sales operations because I really wanted to use all the skills that I had been learning from machine learning, as well as from sales operations in general. So what's the general problem? The general problem is that traditional go-to-market does not scale. There's a few facets to this. The first facet is that teams on the back-office side are either doing work in Excel or Sheets at worst, building analysis each week, multiple hours a week. And as they take on multiple projects, it gets exponentially longer with how many analyses they're doing. At best, they are producing dashboards, providing information to the leadership and executive team, which meets the needs of most teams, but not all of them. And that means that, in general, not all the requirements of the go-to-market teams are met. They're not able to really provide all the information that the teams need when they need it. The second problem, which is more on the go-to-market side with the sales teams and the other teams that I mentioned that I support, is they have two gaps. Essentially, the first gap is the context gap, meaning when a salesperson is talking to a prospect on one call and talking to a current customer on the next call, or talking in an adoption conversation the call after that, they have to constantly switch contexts. And they have to gather information for those specific calls, which is good. They really need to get that information to have those calls and understand how to approach the situation, but they have to do all that work in between. The second one is what I like to call the expert gap, which is the gap between how your expert salesperson or expert go-to-market sales individual would approach a situation, how he would talk to a prospect, how he would work on an adoption call, how he would handle a customer satisfaction issue, and also between a new salesperson or someone that's just ramping up. So ideally, everyone would be working at the same operational level, so you have consistency in execution, consistency in messaging, and consistency in how to assess a customer's problems and how your product can help fit that portfolio. So with these two problems of manual work, as well as salespeople not having enough information and having the gap of having to get all the information they need and gather that, as well as not being able to execute at the same level, it really creates an inefficiency in the go-to-market organization. And so for the last six months or so since I've joined Cloudflare, I've been really focusing on how I can make this operation efficient from back to front. And there's a lot of great things that we've been doing at our company in general that have helped me enable that, and I really want to share some of those findings with you. So the framework that I'm proposing here, which I think is going to really be effective as we flesh this out in the future, is a three-pillar approach. The first pillar approach is how we can scale analysis and the ability of the operations team to meet the data needs of executives, leadership, as well as to be able to build applications using business context. How can they take things that would take two hours to do down to five minutes? In January of this year, after I joined the company, I had built these skills, and I'd started asking questions of the data directly. I was able to get answers immediately while doing other things, and I saw the huge power of how, if we can scale the analysis and the operations of the teams, we can actually focus on the second part of my job, which is strategy. The second pillar is to scale insight. There's a story in the data, and how can we provide that to the team? How can we provide that to the team at the weekly level, at the different levels of management? How can we provide that information, that insight, that story to every customer that these sales teams are talking to? The third one, which is arguably the biggest one, is to provide self-service capabilities to the go-to-market team. When these sales individuals are talking to a customer, how can they get the expert-level information that they need to interact with that customer and to best assess how they should approach the situation, how to handle rejections, how to upsell them, and how to handle customer satisfaction issues? This is a huge part of what I've seen we've done at Cloudflare, and I'll share in a little bit what that looks like. So as it relates to scaling the analytical capability, the back-office operations, what we've done is built role-specific skill files, which have the context of the business information tying it to the data. This is for both technical and non-technical users. Technical users, you could say, are the ones who are building SQL and being able to data engineer a lot of solutions. Then the non-technical people would be more individuals who are closer to the business with the salespeople who may not know how to write SQL. And so we have skill files that they're able to use to ask questions of the data to get answers fairly quickly while doing other tasks. An example here is that in those skill files, through testing, we've included the types of questions that the business would ask of the data, in this case looking at close date changes in opportunities as well as changes in the amount of the opportunities, so that we can answer essentially 80% or more of the questions, where the other 20% might be more complex strategic questions. And so overall, this allows the teams to be able to embed all of the logic into these skill files and get answers fairly quickly. So I've seen users who do not know any SQL, and essentially their requests in the past bottlenecked to someone who knows data and can write SQL for complex queries, be able to just ask questions of the data and get answers. And this is very useful also for what I show later on the third pillar, for building skills for these go-to-market teams so that they can ask questions of their data and get answers. Also, on our team, we've used these same skill files to build multiple applications. When usually that is done in IT and bottlenecked in those areas, we're able to use the semantic information about the business knowledge as well as the columns to be able to build these applications rather quickly. So this allows us to free up our time so that we can focus on strategy and enablement. All right, for the second pillar, what I mentioned earlier, there's a story in the data, and they really shouldn't have to search for it. What I'm showing you here is synthetic data on the right. We have a weekly summary that goes out which highlights how the business is doing, how they're pacing to their goals, and then highlights trends, standouts, as well as watches. So we provide this information to the business so they can, as you can open your phone now on Gemini if you have it and see your notes for the day or the things that you need to do, have that same level of information brought to the go-to-market team so they can just go along their day. And if they do need to look at some of the reports or dashboards, they can drill in, but we bring the story to them. And I'll pull this together why I think this is really important. Of course, there's a place for dashboards and standardized information, but there's a different level of adoption of the KPI metrics at any given company. You're going to have people who love dashboards and people who are never going to look at them. And so I think you really need to have a way to scaffold that across the business. So how do we do this automated analysis? A big part of this is simplifying the data so that the AI agents can actually analyze the data in a very consistent and clean way. Here, what we do is transform the data by the dimension of time, also slice by the logical part of the business, which is manager, theater, and finally, the metric. Here we have data that is wide. You could also go from wide to long. Our trend information that I showed you, but we bring the story to them. And I'll pull this together: why I think this is really important. Of course, there's a place for dashboards and standardized information, but there's a different level of adoption of the KPI metrics at any given company. You're going to have people who love dashboards, people who are never going to look at them. And so I think you really need to have a way to scaffold that across the business. So how do we do this automated analysis? So a big part of this is simplifying the data so that the AI agents can actually analyze the data in a very consistent and clean way. Here, what we do is we transform the data by the dimension of time, also slice of the logical part of the business, which is manager, theater, and finally, the metric. Here we have data that is wide. You could also go from wide to long. Our trend information that I showed you, that data is long, and then we do some preprocessing on that data to then highlight trends. So the embedding of the logic of how you would filter this data to even analyze it, as well as the logical aggregations the business wants to see, is all engineered up front. From my experience, this handles 80-plus percent of the requests, is just getting information about the performance of the teams and how they're doing. You can always go down to the raw data, but this last pillar, which I'll go over in a minute, allows them to do that. To be able to orchestrate this effectively and be able to rely on it, we have a multi-agent workflow where we first get the data, and then we do a first-pass draft on the data, calling our MCPs. And then we have a second reviewer agent who checks the veracity of the data, and then we have a third agent, which is a tone agent who, using a multi-shot prompt, is able to craft the message and highlight the risks and opportunities equally. And with every run, we have observability into each of the LLM calls so we can see what is passed and what is the response that is going on there. And so this architecture, we tested for about two, three months, looking at every single run to see what is going wrong. And this is the model that we had set up that is really working for us. And we really hope to expand this beyond just what I've shown you for multiple teams, but also down to the customer level, as I was just talking to you about. The third part is the self-service model. And what I'm showing you here is our internal tool called Cloudflare OS, which is an agentic workspace that is running on Cloudflare where the go-to-market team can come in here. It spins up their own compute and their own persistent environment using Cloudflare workers as well as durable objects, which is basically storage, sort of like S3. And so the salespeople can come in here and get the data that they need when they need it. So some use cases that these teams are using it for is doing a forecast brief, building QBR decks, building a purchase deck on what the customer that they're onboarding has purchased, doing account planning, general queries of the data, as well as renewal preparation. How are they going to look at what the customers used and either upsell them or figure out how they can get them adopting their product more? So just a little bit more into that Cloudflare OS setup that I just showed you. The AI agent workspace is where that screen I was showing you, and through the three-part piece of the skills in the lower left, which is our expert-level information, as well as the MCP connection and the AI gateway, they're able to have conversations in this agentic workspace to pull data they need and, using expert-level skills, which is curated, they are able to execute the jobs that they need to do when they need to do it. And so a little more information about the skill repository. We have a central alias where skills are presented to the central team, curated by the go-to-market team as well as by the operations team, and they're reviewed so we can make sure that we're not having a proliferation of skills, and we have an expert-level knowledge skill at every level so that they can really get all the information they need for how to approach any customer situation. And so just a few more images here of them using it. Here, we have them building a prescriptive plan for their daily work. They're asking a question. You can see the agent is responding by looking into the MCP and starting to pull the data together. And related to the QBR deck, here's a slide of generating a custom slide deck for a customer call. And so this has really, I think, unlocked the ability of the go-to-market teams to be able to really have all their information serviced to them. And so, bringing it together with the three pillars that I talked about, if you really don't have all these, I think you have issues with serving the go-to-market needs in terms of optimizing the use of agentic systems. With self-service, you allow them to be able to pull data when they need it for whatever situation they need it with the expert-level information. The second is, by pushing the insights to the business, you're able to surface the generalized, standardized information of how these teams are doing, and also so they're aligning with source of truth on performance. And then the third one is the scaling of the analytical team for them to be able to answer queries and build applications for the teams, which really unlocks a lot because the opportunity cost of that team being overloaded and being able to help is that the needs of the go-to-market team are not met. So some findings in the future. So the first thing is skill curation is the basis for all of this agentic workforce. If you're able to embed the knowledge of the business into the skill files as well as the skills for the analyst to be able to build, answer questions, or build applications, as well as those skills that I showed you in the Cloudflare OS, you're really able to give them the ability to use agentic systems in a more predictable and deterministic way so that they can execute evenly across the board. So the second thing is through this whole process, the feedback loop is very important. Just like a company would try to sell a product externally and get feedback, with these internal teams, the feedback loop is very important to be able to see: is what you're building actually useful, what are some issues that they're having, and how can you make this work more efficiently? And the third thing is the layering of those three pillars that I talked about, being able to answer questions where the team comes to you. Some of the go-to-market team, that's how they like to interface with the operations team, is to be able to ask questions. And then the pushing of information, and then self-serviceability. Through that, you're able to interweave all the needs of the team to be able to be met by this agentic run team. So through all these different pillars that I talked about, we've really been able to 2x our efficiency and be able to serve the teams, as well as allowing them to be able to get the information that they need to do their job. And so some things that I see going into the future, number one is a deeper integration with our system that we work in. So, for example, those QBR decks and renewal call skills. How can we set up meetings for the go-to-market team and embed those artifacts in those meetings so they don't have to actually pull them? We can allow that self-service portal to be more ad hoc in what they need. But that requires some information or some security setup, and how can we do that? And then also, another example is getting meeting notes from those calls, which you have to set up across the board. So there's some system-side thing that we have to work on there. The second thing is harder problems around quoting and approvals and updating the CRM itself. We use Salesforce, and we're just in the midst of building the connections and the ability for us to update Salesforce with these agentic systems. And I see that being set up in a way that I set up with the automated analysis, where you have multi-agent workflows to just make sure that everything is getting done right. And the second thing is we've reached the Cambrian stage of using agentic systems, which means there's an explosion of excitement and skills and finding out ways to solve anything with AI. But I see, as we get to this fuller integration and standardization, we're going to want to come back and not really limit, but just figure out a really strategic approach for allowing each team to use the agentic system so that the source of truth and all the systems are aligning. All right. Well, thank you for joining this talk, and I appreciate you coming here. Hope you have a great conference. Around quoting and approvals, and updating the CRM itself. We use Salesforce, and we're just in the midst of building the connections and the ability for us to update Salesforce with these agentic systems. I see that being set up in a way, with automated analysis, where you have multi-agent workflows to make sure everything is getting done right. And the second thing is we're at the Cambrian stage of using agentic systems, which means there's an explosion of excitement and skills and ways to solve anything with AI. But as we get to this fuller integration and standardization, we're going to want to come back and not really limit, but just figure out a strategic approach for allowing each team to use the agentic system so that the source of truth and all the systems are aligned. All right. Well, thank you for joining this talk, and I appreciate you coming here. Hope you have a great conference. yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay you the information that they need to do their job. And so some things that I see going into the future, number one is a deeper integration with our system that we work in. So for example, those QBR decks and renewal call skills. How can we set up meetings for the go-to-market team and embed those artifacts in those meetings so they don't have to actually pull them? We can allow them to that self-service portal to be more ad hoc in what they need. But that requires some information or some security setup and how can we do that? And then also, another example is getting meeting notes from those calls, which you have to set up that across the board. So there's some system-side thing that we have to work on there. The second thing is harder problems around quoting and approvals and updating the CRM itself. We use Salesforce and we're just in the midst of building the connections and the ability for us to update Salesforce with these agentic systems. And I see that being set up in a way that I set up with the automated analysis where you have multi-agent workflows to just make sure that everything is getting done right. And the second thing is we're sort of reached the Cambrian stage of using agentic systems, which means there's an explosion of excitement and skills and finding out ways to solve anything with AI. But I see as we get to this fuller integration and standardization, we're going to want to come back and not really limit, but just figure out a really strategic approach for allowing each team to use the agentic system so that the source of truth and all the systems are aligning. All right. Well, thank you for joining this talk and I appreciate you coming here. Hope you have a great conference. yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay yay you