Intelligence + Continual Learning = Expertise — Yu Su, NeoCognition

Transcript

Music Music Music Music Music Music Music Alright. I understand that I'm standing between you and lunch, so I'll try to be quick. My name is Yusu. I'm a professor at the Ohio State. The Ohio State. And I also have another job, which is CEO at a company called Neocognito. And we focus on agents and continuing learning. So today's talk won't be too technical, but it will be mainly a conceptual one. But I think it's a very important conceptual distinction that I will try to make between what is intelligence and what is expertise. And through this, I will try to answer some of the very bothering questions for me. Why are we so successful at coding agents, but they're so terrible at anything else? Why are the current agents so token inefficient, to the degree that every company right now is coming out and trying to curb their token-maxing efforts in the company? So hopefully this will provide some food for thought before lunch. Right. First, a bit of history. So AI agents are not a new thing, right? We have been trying to develop agents throughout the whole history of AI. But the problem is that in the early stages, let's say in the 1960s to 80s when we developed these expert systems or logical agents, or in the 2010s when we developed these deep neural agents, we were able to capture some very limited facets of human intelligence, right? Whether it's logical reasoning or perception in single modalities to decision. Only recently, with the multi-modal LRMs and the language agents built on top of them, for the first time, we have a neural model that is able to encode multi-sensory inputs into a unified neural representation that is also conducive to symbolic reasoning and communication, right? So that was a trait unique to humans. Now AI agents finally have the same thing. So that drastically improved their expressiveness, their reasonability, and adaptivity. So that's why I think we have really entered a new evolutionary stage of machine intelligence. AI agents. And it didn't take long for these language agents to find their first mass markets, which is coding. And the best way to illustrate this is probably through the revenue graph of Anthropix. Right? In just under two years, their revenue has grown 400 times to 40 billion. I think the newest number is maybe 60 billion annualized runway. And it's largely driven by coding. So then what happens when we leave the privileged world of code? Well, not so well. We are running into a lot of challenges deploying these agents in enterprise settings. And also in personal settings, the open models constantly make quite brittle and silly errors. Then, to the extent that Andrew Kapatty said that it's not going to be the year of agents. It's going to be the decade of agents because they cannot do computer use. They don't have continuing learning. I don't know how much Andrew's thought has changed since last time because of the coding agent and everything. But I think the difficulties with computer use, with continuing learning, are still largely the same right now. So how can something be so smart but also so brittle at the same time? Here's my thesis around it. I think we are actually witnessing a modern version of Moravec's paradox. So Moravec's paradox essentially says that for AI, hard things are easy, easy things are hard. So the modern version here is that we are very good at these symbolic reasoning tasks like coding and math, which were considered crown jewels of intelligence earlier. But then we still struggle with this everyday digital work because they really require quite a different set of cognitive competencies to excel at them. And more specifically, I think modern society is really not just one unified world. It's millions of these micro worlds. Every domain, every profession is different. Every company is different. Even if you are using the same software, every company configures it differently. So it's extremely idiosyncratic, especially in the digital world. It has these unique local physics, different structures, constraints, affordances, and dynamics that you have to learn. It's just too heterogeneous and dynamic for any monolithic model to try to compress it into one static representation. So agents must continually learn on the job to acquire what I call specialized expertise for each specific micro world. The second part of the talk, I will try to establish the differences between intelligence and expertise. Here are the working definitions. For intelligence, it's the capacity to reason through unfamiliar problems from available context. This is what the frontier models are increasingly good at. Give it the problem statements, the context, the tools, and it can reason through this, even if it's seeing them for the first time. And they can do a great job. Every episode is more or less independent from each other here. But expertise is different. Expertise is really accumulated and situated competence. It's the ability to act reliably, efficiently, and with judgment to achieve reproducibly superior performance in a particular domain. Right? So this is in stark contrast with intelligence. And to show what expertise actually contains, I think the key idea from cognitive science is that experts don't just know more facts. They actually see the world differently. Right? So expertise allows you to do different pattern recognition. So you see through the specific patterns. Like if an expert is looking at a gigantic bug report, they can immediately locate the most possible places where things could go wrong. And you think about the problem with a very deep structure. Right? When you are scheduling a meeting, you know that it's not just finding the shared slots on everyone's calendar. It's actually a constraint optimization problem over everyone's authority, the priorities, the urgency, and everything. And experts don't just operate with a set of rules or a set of facts. We know that every single thing is conditional. Right? Every rule has the preconditions where it applies. But we also know when we can bend the rules, when exceptions happen. Right? And finally, that also gives us judgment and taste. Importantly, what's high quality, and very importantly when to stop, when is good enough. So all this together, I think experts effectively have built a world model of their environments. Right? It's a generalized notion of world model that captures how that micro world works. And that becomes the basis for all of our perception, reasoning, decision-making, and judgment. So intelligence and expertise are really quite different across many dimensions. But some of the interesting ones here are intelligence is about, hey, when we have the context, how to solve the problems through the context. But expertise actually will bring you the right context. Right? Given any problem, we know what context is important for this problem and bring it in to solve the problem. And because of that, intelligence tends to expand our search. Every problem-solving is a search problem. So intelligence tends to brute force it. Try to spin up 100 different parallel ways to try to solve the problem. While expertise will actually try to compress the search space. Because expertise has constructed this, has learned these essential shortcuts for the problem space. So that whenever you have a problem, you know the most possible ways to solve it. And then I also think the final part here is that continuing learning is the important bridge from intelligence to expertise. But first, let me try to define continuing learning because it's such a confusing term. And Jack just gave some definition earlier with 10 different names. But here's the definition I work with. I think continuing learning is adaptive compression of experience into reasonable structures for future behavior. So all of these four elements here are very important. For experience, we need to answer the question: what kind of experience we're talking about. Is it more episodes of experience, or is it these semantic facts or procedures or feedback from humans or environments? And how do we compress that? So we embed them into vectors, or we index them into some symbolic structure, we distill them into model parameters, or do some kind of reinforcement learning. And it's not just one-time compression. It needs to be adaptive compression. What you have learned, what you have compressed so far, should largely influence how you compress further. And what kind of structure are we looking at? Is it just parameters, like adapters of your language models, or is it vectors, graphs, skills, or even world models? And then how do you use these reasonable structures? Do you use it just to recall these facts, or use it for prediction of future states? Do you use it for better planning or even for the control, the actuation layer of the agent, or as a value function for potential states? Right? So it's because of this, the continuing learning problem is so rich. It has these four different aspects. And if different aspects can be instantiated in different ways, that makes this field so confusing. But hopefully this is the definition that encompasses most of the versions of continuing learning. Then, I think this is maybe the most important figure in this talk. If we put raw intelligence as the X-axis and expertise as the Y-axis, I think we'll find that they are largely orthogonal to each other. If you don't have continued learning, all you do is scale your model to get better raw intelligence, then what we will get is what I call the world's smartest novice. Right? Super smart. It can try to attack any problem given to it. But it doesn't accumulate expertise. So it ends up just brute-forcing its way at every problem. Then, if you have continued learning, different continued learning algorithms will essentially set the slope of your learning curve here. If you have a sloppy CL algorithm, maybe some kind of simple in-context learning, then with increasing intelligence, your expertise will increase a little bit. But if you have a really strong continuing algorithm, then the expertise will increase rapidly. Of course, this is assuming a given time horizon and experience horizon. And then among all of these potential futures that good continued learning will bring us, I think this is probably the one I like the most, or I think it is the most interesting, which I call unbounded expertise from bounded intelligence. Right? What if we can come up with a continued learning algorithm such that once the right intelligence has crossed a certain threshold, we don't need stronger intelligence anymore? Continued learning will bring us unbounded expertise once we have a reasonable level of intelligence. Right? Then we can call this escape intelligence. And if this is indeed true, then it will have a lot of implications for the whole ecosystem. Right? Do we need to continually train these larger and larger models? Or maybe these models are already good enough. What we're missing is just better continued learning algorithms. So to be a little bit more concrete, I think to provide more food for thought, here are some open questions, I think, in this space. The overarching question is, given any domain or environment, right, how can the agent continually learn to specialize and reach expert-level competency? But to do that, you need to answer many other questions. How do you even define and measure expertise? And this is probably environment-specific. And how to handle the trade-off between reliability and plasticity. Right? We want these agents to be both reliable and plastic. But they are inherently conflicting with each other. Right? Reliable systems, or stable systems, resist change. But plastic systems like to change. So how do we reconcile that? But fortunately, we do have a living existence proof, which is us, ourselves, humans, that we are incredibly plastic, but also manage to be dependable most of the time. Then, from a technical perspective, when we talk about learning, largely there are two forms of learning: parametric or non-parametric. And my belief here is that both are really needed for this type of continuing learning to actually work. But how do we synergize the two? And finally, even though we are focusing on specialization, I think there is a great potential for specialization to actually lead to better generalization. You know we have exhausted the public data for training LLMs. But the next stage of training, the next internet-scale data opportunity, is actually in all of these different private worlds. If we can make these specialized agents work, they can learn in situ and channel back the learning to the general model. I think that may be the next internet-scale data opportunity. So finally, a call to action. Let's start scaling expertise. This will be a new dimension for us to scale. Because intelligence is already becoming abundant. The frontier models, they are probably smarter than average humans. But expertise is still scarce. And we want to build a world where expertise becomes abundant. Where everyone can get expert support. Because in an ideal world, everyone can have their personal health care, personal financial advisor, and personal tutors, and so on and so forth. And then every company can build their own learning loop. They can, as Satya said two weeks ago, we want to enable this human-AI learning loop at each company that turns into institutional memory. And for every company to build their own moats and to still be in charge of their means of production. And finally, I think with abundance of expertise, we will actually see more types of work become possible. Because right now, there are still a lot of opportunities that are locked up. Because the friction is just so high to make them economically viable. But with abundance of expertise, I think that we will be able to lower the friction and make marginal benefits. And we will be able to develop a new type of work across the threshold of worth doing. So this is the future we're building toward at Neocognito, and happy to share this with you. And thanks for the attention. So congratulations congratulations. So congratulations congratulations. So congratulations congratulations. So congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations of the Moravec's paradox. So the Moravec's paradox essentially says that for AI, hard things are easy, easy things are hard. So the modern version here is that we are very good at this symbolic reasoning task like coding and math, which were considered crown jewel of intelligence earlier. But then we still struggle with this everyday digital work because they really require quite different set of cognitive competencies to excel at them. And more specifically, I think modern society is really not just one unified world. It's millions of these micro worlds. Like every domain, every profession is different. Every company is different. Even if you are using the same software, every company configures differently. So it's extremely idiosyncratic, especially in the digital world. It has these unique local physics, like different structures, constraints, affordances, and dynamics that you have to learn. It's just too heterogeneous and dynamic for any monolithic model to try to compress it into one static representation. So agents must continually learn on the job to acquire what I call specialized expertise for each specific micro world. The second part of the talk, I will try to establish the differences between intelligence and expertise. Here are the working definitions. For intelligence, it's the capacity to reason through unfamiliar problems from available context. This is what the frontier models are increasingly good at. It gives it the problem statements, the context, the tools, and it can reason through this, even if it's seeing them for the first time. And they can do a great job. Every episode is more or less independent from each other here. But expertise is different. Expertise is really accumulated and situated competence. It's the ability to act reliably, efficiently, and with judgment to achieve reproducibly super real performance in a particular domain. Right? So this is in stark contrast with intelligence. And to show what does expertise actually contain, I think the key idea from cognitive science is that experts don't just know more facts. They actually see the world differently. Right? So the expertise allows you to do different pattern recognition. So you see through the specific patterns. Like if you're looking, an expert is looking at like a gigantic bug report, they can immediately locate like the most possible places where things could go wrong. And you think about the problem with like a very deep structure. Right? When you are scheduling a meeting, you know that it's not just like finding the shared slots on everyone's calendar. It's actually a constraint optimization problem over everyone's authority, the priorities, the urgency, and everything. And we don't, experts don't just operate with a set of rules or a set of facts. We know that every single thing is conditional. Right? Every rule has like the preconditions where it applies. But we also know when we can ban the reality, we can ban the rules when exceptions happen. Right? And finally, that also give us judgment and taste. It's importantly what's like high quality and very importantly when to stop, when is good enough. So all this together, I think experts effectively have built a world model of their environments. Right? That it's a generalized notion of world model that captures how that micro world works. And that becomes the basis for all of our perception, reasoning, decision making, and judgment. So intelligence and expertise are really quite different across many dimensions. But some of the interesting ones here are like intelligence is about, hey, when we have the context, how to solve the problems through the context. But expertise actually will bring you the right context. Right? Given any problem, we know what context bring into are important for this problem and bring it in to solve the problem. And because of that, intelligence tends to expand our search. Like every problem solving is a search problem. So intelligence tends to brute force it. Try to spin up like 100 different parallel ways to try to solve the problem. Well, expertise will actually try to compress the search space. Because expertise has constructed this, has learned this essential shortcuts for the problem space. So that whenever you have a problem, you know the most possible ways to solve it. And then I also think the final part here is that I think continuing learning is the important bridge from intelligence to expertise. But first, let me try to define continuing learning because it's such a confusing term. And Jack just gave some definition earlier with like 10 different names. But here's the definition I work with. I think continuing learning is adaptive compression of the experience into reasonable structures for future behavior. So all of these four elements here are very important. For experience, we need to answer the question like what kind of experience we're talking about. It's more like episodes of experience or it's like these semantic facts or procedures or feedback from human or environments. And how do we compress that? So we embed them into vectors or we index them into some symbolic structure. We distill them into model parameters or do some kind of reinforcement learning. And it's not just like one-time compression. It needs to be adaptive compression. Like what you have learned, what you have compressed so far should largely influence how you compress further. And what kind of structure we're looking at? It's just like parameters, like adapters of your language models or it's vectors, graphs, skills, or even word models. And then how do you use these reasonable structures? It's like you use it just to recall these facts or use it for prediction of like future states. You use it for better planning or even for the control, like actuation layer of the agent or as a value function for potential states. Right? So it's because of this, the continuing learning problem is so rich. Like it has these four different aspects. And if different aspects can be instantiated in different ways, that makes this field so confusing. But hopefully this is the definition that encompasses most of the versions of continuing learning. Then, I think this is maybe the most important figure in this talk. If we put raw intelligence as the X axis and expertise as the Y axis, I think we'll find that they are largely orthogonal to each other. If you don't have continued learning, all you do is scaling your model to get better, like raw intelligence. Then what we will get is what I call the world's smartest novice. Right? Super smart. It can try to attack at any problem given to it. But it doesn't accumulate expertise. So it ends up just like brute forcing its way at every problem. Then, if you have continued learning, like different continued learning algorithms will essentially set the slope of your learning curve here. If you have a sloppy CL algorithm, maybe some kind of simple in-context of learning, then with like increasing intelligence, then your expertise will increase like a little bit. But if you have a really strong continuing algorithm, then the expertise will increase rapidly. Of course, this is assuming like a given time horizon and experience horizon. And then among all of these potential futures that good continued learning will bring us, I think this is probably the one I like the most, or I think it is the most interesting, one I like the most interesting thing, which I call the unbounded expertise from bounded intelligence. Right? What if we can come up with a continued learning algorithm such that given up, once the right intelligence has across a certain threshold, we don't need stronger intelligence anymore. Like, continued learning will bring us like unbounded expertise once we have like a reasonable level of intelligence. Right? Then we can call this the escape intelligence. And if this is indeed true, then it will have a lot of implications for the whole ecosystem. Right? Do we need to continually train these larger and larger models? Or like these models like maybe they're already good enough. What we're missing is just like better continued learning algorithms. So to be a little bit more concrete, I think to provide more food for thought, here are some open questions, I think, in this space. The overarching question is like given any domain or environment, right? How can the agent continually learn to specialize and reach X level competency? But to do that, you need to answer many other questions. Like how do you even measure, define and measure expertise? And this is probably environment specific. And how to handle the trade off between reliability and plasticity. Right? We want these agents to be both reliable and plastic. But they are inherently conflicting with each other. Right? Reliable systems or stable systems, they resist the change. But the plastic systems like to change. So how do we reconcile that? But fortunately, we do have a living existence proof, which is us, ourselves, humans, that we are incredibly plastic, but also manage to be dependable most of the time. Then, from a technical perspective, like when we talk about learning, largely there are like two forms of learning, parametric or non-parametric. So how, and my belief here is that both are really needed for this type of continuing learning to actually work. But how do we synergize the two? And finally, even though we are focusing on specialization, I think there is a great potential for specialization to actually lead to better generalization. You know we have exhausted the public data for training LLMs. But the next stage of training, the next internet scale data opportunity is actually in all of these different private worlds. If we can make this specialized agent work, they can learn in situ and channel back the learning to the general model. I think that may be the next internet scale data opportunity. So finally, a call to action. Let's start scaling expertise. This will be a new dimension for us to scale. Because intelligence is already becoming abundant. The frontier models, they are probably smarter than average humans. But expertise is still scarce. And we want to build a world where expertise becomes abundant. Where everyone can get expert support. Because in an ideal world, everyone can have their personal health care, personal financial advisor, and personal tutors, and so on and so forth. And then every company can build their own learning loop. They can, as Satya said two weeks ago, like we want to enable this human AI learning loop at each company that turns into institutional memory. And for every company to build their own modes and to still be in charge of their means of production. And finally, I think with abundance of expertise, we will actually see more types of work become possible. Because right now, there are still a lot of opportunities that are locked up. Because the friction is just so high to make them economically viable. But with abundance of expertise, I think that we will be able to lower the friction and make manual benefits. And we will be able to develop a new type of work across the threshold of worth doing. So this is the future we're building towards a new cognition. And happy to share this with you. And thanks for the attention. So congratulations congratulations. So congratulations congratulations. So congratulations congratulations. So congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations congratulations