SPEAKER_02
Can you define EBM for us?
SPEAKER_03
EBMs are naturally non-autoregressive. There are no sequences of tokens, and that's what makes it fundamentally different. Imagine you're trying to navigate a map, and you have an LLM. To navigate, you're allowed to choose one direction at a time, and sometimes you take the wrong turns just because you hallucinate. There might be a hole in the road, and you're just going to fall. And you might see this hole, but you cannot turn back because you're an autoregressive LLM. EBM is going to have the bird's eye view all the time. So if you see there's a hole, you're going to choose a different route. [SPEAKER_02] Eve, welcome to the show. Hi, thanks for having me.
SPEAKER_03
[SPEAKER_02] Great to have you on. For people who don't know, you are the founder and CEO of Logical Intelligence. Tell us what Logical Intelligence does.
SPEAKER_03
So Logical Intelligence does a few things. First of all, we see ourselves as a foundational AI company. So we work with both EBMs and LLMs. Everything that's built in-house, we prototyped on LLM initially, and we're building EBM at the same time, and that gets plugged in in the long term. We focused on correctness of software and hardware as a product because I believe there are a lot of issues with AI being placed in mission-critical systems today. There's a big gap on the market today, having deterministic AI, verifiable AI. So we're trying to fill that gap.
SPEAKER_03
[SPEAKER_02] The place my brain goes first is why does correctness or whether something makes sense, why does that matter if it works? Actually, let me ask you a question back. So speaking of correctness, I don't know. Well, imagine there's AI driving a car and you are in that car and that car is an LLM. And someone tells you like 20% of the time it's going to hallucinate and you might end up in the wrong place. How would you feel about it? [SPEAKER_02] Well, I think in my case, I'd be like, wow, that's interesting. I'm curious where it takes me.
SPEAKER_03
Oh, okay. Yeah. Let me give you another example. Yeah, sure. How about the plane? How about the plane? You take a plane from SF to New York and someone says 20% of the time it might just not go to the right place and it's going to go down. So how would you feel about it? [SPEAKER_02] Yeah. My feeling about that is planes are currently run very well by deterministic systems. So I don't know why I would need an AI for that.
SPEAKER_03
I feel we just cannot avoid AI anywhere. Next 10 years, people are going to try to place AI everywhere, automate systems with AI. And technically you might not need it, we survived somewhat without AI up to this moment, but now it's just a next step of evolution that people just want AI everywhere. For banking, you don't need AI initially, but we learn it's really helpful to automate certain processes in decision making. And it's going to save us a lot of time and allow space to be creative instead of debugging and fixing things. So I just feel it's an unavoidable future.
SPEAKER_03
[SPEAKER_02] I think maybe what I'm getting at is, what am I getting at? It seems like if you want a guarantee of certainty using, the only way to guarantee certainty is to use something that you can express in code or logic.
SPEAKER_02
[SPEAKER_03] That's a part of it. So the certainty comes from internal verifiers and external verifiers, at least for us. So for example, if you take an LLM, obviously it's a language-based model and the architecture doesn't allow you to do internal verifiers. So it's a black box for you. You don't have access to what's inside until it's all processed. But you have access to the output and many people and companies sort of take LLMs trained for certain tasks. And if it requires logic, they attach external verifiers to it, such as languages like Lean4, which is a proof language, a machine-verifiable language, which allows you to check this output using mathematical frameworks. However, you know, it doesn't solve the problem of things being expensive. Because what's expensive is your architecture, which is still playing a guessing game. And even if you attach an external verifier, even if you fine-tune this LLM specifically for the task you're trying to create, you're still not solving the problem of tokens being expensive. It takes compute for you to play a guessing game. So this problem is solved by EBMs, but we're talking about LLMs for now. So here we have the situation when there's an internal absence of verifier, but there's an external one. So now about EBMs, EBMs don't have tokens. It's a token-free model.
SPEAKER_03
However, it doesn't solve the problem of things being just so expensive.
SPEAKER_02
[SPEAKER_03] Because what's expensive is your architecture, which is still playing a guessing game out here. [SPEAKER_03] And even if you attach external verifier, even if you fine tune this LLM specifically for the task you're trying to create, you're still not solving the problems of tokens being expensive. [SPEAKER_03] It takes compute for you to play a guessing game.
SPEAKER_03
So this problem is solved by the EBMs, but we're talking about LLMs for now. So here we have the situation when there's internal absence of verifier, but there's external one. So now about the EBMs, EBMs don't have tokens. It's a token-free model. There's no guessing game of this kind. So essentially, you could oversee all the possible scenarios. [SPEAKER_02] Can you define EBM for us? Yeah, I'll define in a second.
SPEAKER_03
So for now, just think of it as something which doesn't play a guessing game. And something which has architecture which essentially allows you to self-align itself as processing the information. And it's no longer a black box for you.
SPEAKER_02
[SPEAKER_03] So as it's performing, you can open it anytime during the training and you could see what's happening in there. [SPEAKER_03] So you cannot do this with LLMs.
SPEAKER_03
Just the nature of architecture is different. So you have for verification tasks, you have this notion of self-alignment because of the EBM architecture and the absence of tokens makes it cheap, but also you have external verifier on top of it. So you have verification on both sides, inside and outside. Hopefully that makes sense. [SPEAKER_02] I think so. [SPEAKER_02] Let me play it back to you and you tell me if I'm getting you.
SPEAKER_02
So I think what you're saying is we're living in this world, which is really cool with LLMs, which is we can generate lots of output with them. And the output is really useful for a lot of different things.
SPEAKER_03
[SPEAKER_02] But in order to tell if the output is right, the best we can do is guess and check. [SPEAKER_02] We generate the output. [SPEAKER_02] And then, for example, if it's code, then we go and check the code with integration tests or manual tests or whatever, just to see if it works. [SPEAKER_02] And that totally works, but it is expensive and time consuming. [SPEAKER_02] And one of the problems is it's very hard for us to know, okay, how did the LLM get to this answer? [SPEAKER_02] We can't go look inside of it. Exactly.
SPEAKER_03
[SPEAKER_02] And I think what you're saying is there are other types of models that are a little bit more inspectable and that give us a sense before we even try the output to understand, does this work? [SPEAKER_02] Does the output work?
SPEAKER_02
We can get a sense from the model by looking at its internals. Sort of like, how good is this solution? How good does this model think the solution is?
SPEAKER_03
[SPEAKER_02] And it's like being able to ask someone, are you sure about this? [SPEAKER_02] How good is this before you go check their work? [SPEAKER_02] And a language model can answer that question. [SPEAKER_02] But a language model's answers are working at a different level when it answers that question than these EBM models are working. [SPEAKER_02] And the answers from EBM models are more likely to be correct. Yeah. So you always have an opportunity to see what's inside with the EBMs and you control the training. EBM, sorry. [SPEAKER_02] Yeah. So the EBMs, you control the training.
SPEAKER_02
[SPEAKER_03] It's no longer a black box for you.
SPEAKER_02
[SPEAKER_03] You control how the training goes. [SPEAKER_03] Well, you do to some extent with the LLMs, but you need to wait until the training is done before you actually go and see what's inside.
SPEAKER_03
In here, you could do it in real time. Yeah. And also you can attach the same external verifiers, which works for LLMs. So you have double verification. Yeah. So you asked me, what is the EBM? I just want to give a historical note because I feel there are so many terms today and people throwing those terms without defining them. So EBM simply means energy-based model. What is energy? Energy-based. It comes from physics. It's a very popular term when they're trying to minimize the energy. And if you're doing theoretical physics, your full-time job is just to write Lagrangians, which correspond to terms associated with the energy in your system. This is my kinetic energy.
SPEAKER_03
This is my potential energy. And then you're trying to derive equations of motion. And the way you derive the equations of motions is by doing the minimization. So that's pretty much how whole theoretical physics works. Start with the energy terms. Then you minimize this energy and you derive equations of motions. And equations of motions are going to give you conservation laws. So you're going to know exactly what the laws are about your system.
SPEAKER_02
[SPEAKER_03] And this principle is a fundamental principle.
SPEAKER_03
Everything wants to minimize energy around us. Yeah. So even us, we're talking to each other, we're sitting on chairs, we're not jumping and running around because it's a natural state when we minimize the energy. So we're just using this minimization energy principle as AI is processing information in high-level terms. So the term energy-based minimization doesn't really mean anything specifically to AI. It's just the whole idea of, let's take some energy and try to minimize it and discover what the laws about it are. So our model is called the official name of that model.
SPEAKER_03
Even though we call it Kona, just because we like big fans of coffee culture and Kona is one of our favorite kinds. So we decided to start with that. The formal name of the model is called energy-based reasoning model with latent variables. And I'm going to describe exactly what these words mean. So we already understand what the energy minimization is. [SPEAKER_02] Can I actually pause you? [SPEAKER_02] Because I want to make sure that we do understand what the energy-based minimization is.
SPEAKER_02
Okay. [SPEAKER_03] Yeah. For now, think of it as just something which minimizes the energy. [SPEAKER_03] It means this AI architecture has a framework, which allows you to construct the energy function of your system and minimize it. I get it. I just want to make sure for people listening, they understand what [SPEAKER_03] And I'm going to describe exactly what this word means. [SPEAKER_03] So we already understand what the energy minimization is. [SPEAKER_03] Can I actually pause you? Because I want to make sure that we do understand what the energy-based minimization is.
SPEAKER_02
Okay. [SPEAKER_03] Yeah.
SPEAKER_03
[SPEAKER_02] It's just for now, think of it as just something which minimizes the energy.
SPEAKER_02
[SPEAKER_03] It means this AI architecture has a framework, which allows you to construct the energy function of your system and minimize it. [SPEAKER_03] I get it. I just think that so I just want to make sure for people listening, they understand what it means to minimize energy, what energy is and what it means to minimize it. So I'm curious, give me a tell me if this concrete example is about what you're talking about. So if I'm going to, let's say, I'm going to go lie on the couch behind me and I'm trying to predict or understand how is my body going to be lying on that couch? Given the laws of gravity, the couch is uneven, my body's uneven.
SPEAKER_02
And so I'm trying to understand the fit of how my body is going to end up settling onto that couch. I'm going to end up setting onto the couch in a way that minimizes energy. So there's going to be a good fit between my body and the couch versus me being jerky like this and having lots of different spaces. Is that the sort of energy minimization that you're talking about? Yeah. [SPEAKER_03] Yeah.
SPEAKER_03
You, it's all about your body finding the most comfortable configuration for you, which is going to correspond to the lowest potential of your body. I would even tell an even more high level example of this. You, Dan, just imagine you're tired. You're done thousands of podcasts and you just came home and someone is asking, okay, Dan is a variable here.
SPEAKER_02
[SPEAKER_03] Let's try to figure out what's his equations of motion in the house and where he's going to most likely end up.
SPEAKER_03
So you're probably going to end up on the couch. With a nice show and probably some drink. [SPEAKER_02] Yeah. So that's going to be a law, okay. When Dan is tired, he's going to go and sit on the couch and just relax. But to get there, we're going to look at all your possible states, washing the dishes, walking around the house. Those are going to be different states, but your most probable scenario is going to be on the couch. So essentially all of this picture can be mapped into something we call energy landscape. It's going to have the highest points. It's going to have the lowest points. The highest points we can associate with less probable scenarios.
SPEAKER_03
So probably if you're tired, you're not going to dance around. Although I don't know, but typically people assume that if you're tired, you're probably going to want to relax. So that's going to be the lowest point. And as we're trying to figure out where you are during the training, we're going to observe you multiple times during different days. And how much of the workload you have is going to be a variable. Your internal state is going to be a variable, how your body feels. And eventually we're going to train this landscape to be based on what we see in the real world, right? The lowest point is going to be you on the couch. We've all been there.
SPEAKER_03
[SPEAKER_02] You're sitting in an important meeting and you're trying to pay attention. [SPEAKER_02] You're trying to stay present, but you have this lingering underlying anxiety that you're going to forget everything, that you're going to miss the important detail, forget the decision, forget the action item, let something important slip through the cracks. [SPEAKER_02] That's why I love Granola. [SPEAKER_02] It's an AI powered notepad that works in the background while you're in your meetings. [SPEAKER_02] It takes notes on everything that gets said, transcribes action items, and helps get rid of that feeling.
SPEAKER_03
[SPEAKER_02] You don't have to worry about whether you're going to miss something because Granola has you covered. [SPEAKER_02] And that lets you stay present in meetings. [SPEAKER_02] I've been using Granola for a long time, almost since they came out. [SPEAKER_02] And it's amazing for this. [SPEAKER_02] It doesn't join the meeting like some of those other clunky meeting note takers. [SPEAKER_02] The UI is really fast and well considered. [SPEAKER_02] And it feels like it's transcribing all the important moments in my work life. [SPEAKER_02] And that gives me the confidence to get great work done.
SPEAKER_03
[SPEAKER_02] And what's even cooler is you can chat with your notes afterwards. [SPEAKER_02] You can run detailed research reports on how your week was, how you act as a leader, how you performed in particular difficult conversations and how you can do better. [SPEAKER_02] It's a power tool for anyone who cares about their meetings and also cares about how they show up in those meetings. [SPEAKER_02] It also has these things called recipes, which are pre-made prompts for common tasks like negotiating, coaching, or summarizing. [SPEAKER_02] I even have a recipe that I made that's in Granola that you should check out.
SPEAKER_03
[SPEAKER_02] Once you try it on one meeting, it's really hard to go back. [SPEAKER_02] The notes are always better than what you could do manually. [SPEAKER_02] And it helps me be much more present instead of frantically typing all the time. [SPEAKER_02] Head to granola.ai slash every for three months free with the code every, E-V-E-R-Y. [SPEAKER_02] That's granola.ai slash every for three months free. [SPEAKER_02] And now back to the episode. [SPEAKER_01] Okay. [SPEAKER_02] That makes total sense. [SPEAKER_02] Now I want to relate this to LLMs for a second, because you can imagine that there's an LLM that's trained to predict where I end up after a long day of podcasts.
SPEAKER_02
And you can imagine it probably would also end up predicting that I would end up on a couch. What are the differences in the ways that it makes those predictions that make energy-based models better for this scenario?
SPEAKER_03
[SPEAKER_02] Okay.
SPEAKER_02
[SPEAKER_03] That's a good thought exercise.
SPEAKER_03
So, okay, now you are an LLM. [SPEAKER_02] And now back to the episode. [SPEAKER_01] Okay.
SPEAKER_02
I, that makes total sense. Now I want to relate this to LLMs for a second, because you can imagine that there's an LLM that's trained to predict where I end up after a long day of podcasts. And you can imagine it probably would also end up predicting that I would end up on a couch. What are the differences in the ways that it makes those predictions that make energy-based models better for this scenario? Okay. [SPEAKER_03] That's a good thought exercise.
SPEAKER_02
[SPEAKER_03] So, okay, now you are LLM. So they, okay, let's talk about back to EBM, because what we described is very natural about EBMs. EBMs are all about constructing energy landscapes and how we navigate those energy landscapes. And energy landscapes is the maps of your states based on the data we observe. So in your case, we're just going to look at you in all possible scenarios. All of these possible scenarios are going to be mapped into energy landscape, highest point—less probable scenario, lowest point is more probable. So you're going to be— Very probable and on the couch. Yeah, yeah, yeah. Um, there might be some other additional low points.
SPEAKER_02
[SPEAKER_03] Like sometimes you might go to gym. [SPEAKER_03] Right on the couch. [SPEAKER_03] Like you feel tired and you might go to gym. [SPEAKER_03] So it's going to be, lowest points compared to everything else, but some of them are going to be lower.
SPEAKER_03
Yeah. [SPEAKER_02] Um, yeah. So that's the situation. So this is how energy based model actually we think. It just takes the data and maps it directly to this energy landscape. And then we use certain algorithms to navigate this, but there are different kinds of energy based models today. So I'm going to talk about it a little bit later, but the whole idea is just, hey, let's map it into the structure and navigate the structure.
SPEAKER_02
[SPEAKER_03] Hmm.
SPEAKER_03
And as you see, as we map into this, there are no tokens. We don't predict any tokens and so on. So that's already a crucial difference. How would LLM think? Um, LLM. Yes. It's going to rely on the training data and it's going to be a lot of training data. Like a lot, a lot of observations of how you behave. And to figure out where you would end up, it's going to be attached to probabilities of your next token.
SPEAKER_03
Yeah, if that makes sense. And those tokens are going to come from words.
SPEAKER_02
[SPEAKER_03] And what usually bothers me about LLMs is intelligence, which is language dependent. Like our brains, we are intelligent. I'm relatively intelligent. So I speak different languages and none of my thought processes really depend on any language. Like I could just think in an abstract way and then I speak different languages and decode the information in the channels. And with LLMs, if you're searching for the next token in certain words, the intelligent process, I would say the information processes in French are going to be different from what's in English. Just because words naturally are going to end up next to each other. Do you see what I'm saying? And then we have so many languages in the entire world. And you have so many LLMs trained on different languages. So you're going to end up reasoning. You're going to end up having reasoning processes different for each of the languages, which feels really wrong. So in this case, observing you walking around the house has nothing to do with language then. It's a pure visual spatial reasoning task. It's just looking at your body, navigating the space time and geometry of your house. So we need to map that information in the language space, find the right words and embeddings. And then we start associating those tokens with the probabilities based on what the data we see from you. So we're trying to map something that absolutely has nothing to do with language into language space and think about it in that space, which feels really wrong. And I don't know. I'm just realizing that for many people, it's counterintuitive just because LLMs is the first form of AI we know, and it's the most popular form of AI today. Like for many people, it's by default, like, oh yeah, we're just going to use language to navigate the world, to drive a car. But every time I'm speaking, I'm like, well, let's wake up. Let's actually see when you drive a car, when you walk around your house, how much language you actually use. Are you trying to predict next word as you navigate yourself around the house? Probably not. You just use your visual data, your state of the body, and you just move your body, right? Without speaking.
SPEAKER_03
There's a lot here. [SPEAKER_02] I'm really into this conversation.
SPEAKER_03
[SPEAKER_02] So I want to start with, A, it seems absolutely right to me that there are many different ways in which we process information or many different ways in which intelligence can occur and only a few of them are verbal. But there's certain things that come up for me when I think of this. One is language models happen to work with languages as their primary way of working. But really, they just work with sequences of tokens that have weak correlations, many thousands of weak correlations between each token that helps us to know which comes next. So even though it might be unintuitive to model my behavior inside of my apartment with specifically with language, although I think there is some interesting things there that it might be related to language, we could model it as just like a sequence of movements, right? Where one movement is weakly correlated to the next one that we sort of have a trajectory of movements that tell us where I'm going. Why is that not a good way to model things?
SPEAKER_03
It's a good way to model things and you don't need a lamp for it. You need a form of AI which is not attached to language, but it can be compatible with language if you want it to. And that's what our model is about. [SPEAKER_02] Right. I guess what I'm saying is forgetting about the language part of it, just like modeling my movements as this string of correlated events, like an event stream where each token is like one next thing I do. Yeah, you can do it and people do it today, right? People even do image recognition using language models. You could be really creative, but it's like that's what makes it expensive and super slow.
SPEAKER_03
[SPEAKER_02] It's a good way to model things and you don't need a lamp for it. [SPEAKER_02] You need a form of AI which is not attached to language, but it can be compatible with language if you want it to. And that's what our model is about. Right. I guess what I'm saying is forgetting about the language part of it, just modeling my movements as this string of correlated events, like an event stream where each token is like one next thing I do.
SPEAKER_03
[SPEAKER_02] Yeah, you can do it and people do it today, right? People even do image recognition using language models. You could be really creative, but it's that what makes it expensive and super slow because you're trying to play a guessing game what my next token could be. And this is what makes it extremely expensive. So you could do it, but you don't have to do it. You just can use different architecture which is more suitable for non-language related tasks such as spatial reasoning or applied engineering is another example of spatial reasoning. Like when you build a bridge, you don't go to literature department, you go to engineering school and learn formal methods, right? So here we are trying to use literature department everywhere. And I'm like, Hey, we don't have to, there are EBMs. There are also other forms of AI, which you can experiment with and you don't have to do everything through language.
SPEAKER_03
It's a matter of it's right. It's energy-based minimization principle when it comes to your resources. If you have infinite money and you don't care about the time scale, sure. You can do everything. You can attach it to language. You can attach it to, I don't know, your cat movement around the house and connect it to the cat movement and you know where cat goes and your next token goes and we decide where you're going to go. You can be really creative, but if you want to minimize your resources and you don't have opportunity to wait. Like for example, if your AI controls the circuits, you probably cannot wait even a second. It's all milliseconds, microseconds. So it's just, this form of AI is not suitable for those tasks.
SPEAKER_03
So basically if I'm understanding this right, if I'm spending tons and tons of tokens and I'm looking for a more efficient, more direct way to predict some of these solutions to these problems, an energy-based model is going to get me there faster than modeling it with tokens. Is it also able to do with less training data? Yes. Actually the beauty of the EBMs is it's really good at working with sparse data because there's evolutions of traditional EBMs, which were applied for the LLMs. Then there was diffusion models and diffusion models came from the fact that sometimes you don't have enough data to train the models or your data set is just incomplete. So there are ways to reconstruct those energy landscapes by injecting certain noise and changing the navigation strategies. So that's what the diffusion models were about. And the EBRM with latent variables is just like, hey, on top of the diffusion stuff, we also understand the data. We're not just taking any data, but we also understand why the data looks the way they are. So that understanding goes to the latent variables, just like latent space in your brain understands the world around you and keeping you on top of your tasks and allows you to predict and plan. So it was the same idea here.
SPEAKER_03
So now we got to the latent variable part of our part of it. So I would love when you use the word understanding, I think that must mean something very specific to you. Can you help me understand that and how it relates to latent variables and what those are? Yeah. So that's also back to your question was that like how LLMs are different from those kinds of EBMs we're creating. LLMs don't understand the data. It's just, you feed a lot of data into it and it's like, hey, I got it. I know what's the most probable scenario here and here we are. However, here EBM, you can feed a lot of data. It's not just going to look at like, hey, I see the biggest pattern here. It's going to try to understand the pattern and that understanding, that knowledge is going to go to latent variables.
SPEAKER_03
[SPEAKER_02] So what is understanding about data? It's just basic knowledge about the world, basic rules about the world. Like if there is a couch behind Dan, it's probably because he likes to sit on it or because he likes it on the background. So there are little rules you can guess about you being a data point and your couch being data point. And then there's, you can try to create those kinds of rules for everything, right? For you navigating your apartment, there are little rules, there's a kitchen for cooking. There's, I don't know, bathroom, there's sofa, there's your bed. So that understanding allows you to have your own mental world model for your brain, which helps you to understand your environment. And if something changes in your environment, you understand the rules. Like if somebody brings you a different couch, different shape, you're still going to know what to do with it. So that's an example of how you can infer what to do with something new based on what you already know. So with people, it comes natural because of the evolution and so on, but with AI, we need to teach it. So we need to mimic that evolution. And what latent variables allow you to have here is like, Hey, let's look at the data, but let's also try to understand the data. Let's look at, you know, if you do numerical analysis, we're going to look at all possible correlation functions. And the model is going to be creative. It's going to try to figure out what's the total state of the energy and minimize and figure out the laws about your data. But there are so many creative ways how you can infer those rules. What is a, so is a latent variable equivalent to a rule in this scenario? Like if there's a couch in my apartment, I sit in it. It's not equivalent to a rule, but it's equivalent to something which holds the knowledge about the rules of your data. It's like a knowledge storage. So it has many rules in it? Yeah. You could have many rules. So one latent variable has many different rules. Yeah. It's just like a knowledge data set essentially about your data. Is it an explicit data set as in like, does it have key value pairs of rules or is it a, it's in a form of energy landscape. It's just another analogy landscape you're going to navigate. So essentially we take the data, we look at the data, we construct some sort of structure for AI to deal with the data. So it could start learning the rules about the data. And once it understands the rules, it stores its knowledge in the latent variables in the form of energy landscape. And then we navigate that energy landscape later. Interesting. And like, could it, for example, explicitly write out for me, theoretically explicitly write out for me, here are all the rules that I know,
SPEAKER_03
explicit data set as in does it have key value pairs of rules or is it in a form of energy landscape. It's just another analogy landscape you're going to navigate. So essentially [SPEAKER_02] we take the data, we look at the data, we construct some sort of structure for AI to deal with the data. So it could start learning the rules about the data. And once it understands the rules, it stores its knowledge in the latent variables in the form of energy landscape. And then we navigate that energy landscape later. Interesting. And could it, for example, explicitly write out for me, theoretically explicitly write out for me, here are all the rules that I know.
SPEAKER_03
are all the rules that I know. Or is it, it stores all of them in this energy landscape, but. Yeah, we can access that. We can access that. And that's what makes it, that's what EBM potentially makes it powerful for data analysis because data analysis is all about searching for patterns and rules about your data. So it's something where language is not going to be helpful to you. If you try to attach the rules about your data and those data is numbers and some relationships and functions to American English and words in American English, and then you try to search for the next word that you're losing a lot of information. So in this case, you have an opportunity to directly work with the data and understand the data. Yeah. I think one of the things I'm trying to understand is when I hear rules about the world and how things relate to each other, I think of symbolic AI. And I'm wondering, and obviously those approaches ended up being pretty brittle and requiring too much compute and stuff. And I'm wondering how an energy landscape that acts as that is, that stores a bunch of rules about the world doesn't fall into the same problems. Well, because I guess we avoid tokenization in this case, we just map it directly into different data structure. So EBMs are naturally non-autoregressive, there are no sequences of tokens and that's what makes it fundamentally different. So essentially, I don't know if it helps, there could be another analogy. Like you're trying to navigate the maze and you are LLM person. So you have LLM brain. Maybe maze is not a good example. Imagine you're trying to navigate the map of San Francisco. And you have LLM brain. So you're like, okay, I'm in mission bay. Let me turn to Embarcadero. So you cannot choose. So essentially you were forced to choose one direction at the time. So you choose to walk to Embarcadero and you're just going to keep walking and walking. And if you want to turn, you just need to choose one direction at the time. And imagine you're trying to get to the Bay Bridge from King street. It's typically a 20 minute walk, but depending how you walk. So to navigate there, you're allowed to choose one direction at the time. You don't see any other options. You have tunnel vision and you just keep walking, one decision at the time. And sometimes you take the wrong turns because you hallucinate, some words just naturally next to each other. And it doesn't allow you to turn right when you want to turn left. And then you just keep wandering and wandering until you try to reach the Bay Bridge. And the roads you're taking might never take you there. There might be a hole in the road and you're just going to fall, but you might see this hole, but you cannot turn back because you're an autoregressive LLM. You have to go into that hole. And that's how sometimes you run out. So this is the reason why sometimes we prompt it and it doesn't give you an answer. It just because it's searching and searching and searching, it's spending more and more compute and it doesn't have a bird's eye view. It just doesn't have ability to turn as it performs the tasks. It doesn't know what's right and what's wrong anymore. It just randomly chooses one direction at a time and keeps walking until you try to reach it. So you might never reach that destination. And that's why you need a lot of training. So how is it different from the EBM? EBM is going to have the bird's eye view all the time and you're allowed to take different routes. So if you see there's a hole, you're going to choose a different route.
SPEAKER_03
It may not look like it, but Dan Shipper is currently hard at work testing the latest Claude's Opus models. Working looks pretty different in this new world. We call this Hammock Mode. [SPEAKER_02] Hammock Mode's over. Looks like Dan has to jump in. [SPEAKER_02] Hammock Mode, an idea by Every. [SPEAKER_02] Every, the only subscription you need to stay at the edge of AI.
SPEAKER_02
[SPEAKER_03] That's really interesting. It basically, I've been doing a lot of coding with language models recently to test the limits of vibe coding. And one of the things that I find, or have found with big production apps is in particular, if you have vibe coded something, over the course of vibe coding it, you may have slightly changed exactly what is this project even supposed to be about and what are the problems that I'm trying to solve with it. And if you then go look at the code base, it feels like all of the code is locally correct, but it forms this sort of patchwork of hot fixes and solutions where if you zoomed out, you'd be like, actually, there's a much simpler way to think about how to do all this stuff.
SPEAKER_02
But it has trouble when it's presented with a lot of context, then zooming down into, okay, I need to create a unified solution here that is not a patchwork of different things, but carries one concept throughout the entire system. And it ends up being distracted a lot by whatever it's looking at at the current moment. Is that the sort of problem that you think this type of system can help with?
SPEAKER_02
There's actually a lot of problems in what you're describing. So solving the problems with vibe coding is one of our use cases, we're dreaming about generating formally verified code and automating the coding entirely. So moving you from vibe coding in one specific language to coding in natural language, so you can call natural English, for example, and no more C++ or Python. It needs to be involved in there. So that's an idea. And with coding as it is today, yes, we prompt LLMs and it gives us something back, but it's still on you, as an engineer to figure out what's right and what's wrong. So there's going to be a set of rules. LLM can try to help you. And even if it has external verifier, which is going to check whether your old logic is in your GitHub,
SPEAKER_02
[SPEAKER_03] Speaker 1 It needs to be involved in there. So that's an idea. And with the coding is, at the state it is today. Yes, we prompt LLMs, and it gives us something back, but it's still on you as an engineer to figure out what's right and what's wrong. So there's going to be set of rules. LLM can try to help you. And even if it has external verifier, which is just going to check whether your old logic in your GitHub space is compatible compliance to what are you trying to create? And if the new logic is compatible with your old logic. So this thing's external verifiers can check. They could just say, Hey, we know the old logic. We know the new logic.
SPEAKER_02
Speaker 1 We're going to see how it's merged together. We're going to write mathematical proof, making sure that this logic is compatible with what you already have, and provide you a certificate. It's all machine verifiable. It's all happening on compiling level. So all it's going to say, it's going to send you a message in natural language like, Hey, look, this part of your code is not compatible by logic.
SPEAKER_02
[SPEAKER_03] Speaker 1 This is potentially how you fix it. And this is the things we cannot fix for you. So we moving you from vibe coding to vibe code specifications, those rules and information about your code is called code specification. So once you understand, this is the first problem, right? We're trying to solve. It's just logic can be incompatible with what you already have.
SPEAKER_02
[SPEAKER_03] Speaker 1 The second problem is, is this code actually doing what you want it to be doing? And this is what AI cannot solve for you. Because AI cannot look in your brain and know what you want. Example would be, imagine you're coding vibe coding autopilot. So you have specifications from the hardware perspective. You have specifications from your logic perspective. Like, Hey, make it. And there's also instructions, right? How the car is supposed to behave. So there's behavior parameters for your code. So code being able to be compiled is one problem. The second problem is, is this code doing what you want it to be doing? So for example, how fast it is on the hardware and so on. And if the answer is yes. Another set of questions is, okay, is it going to hit the pedestrian by chance? Is it actually going to navigate the map of, I don't know, San Francisco? And the answer is, I don't know, right. So in here you need to write a bunch of tests and test your entire system. You create it like, Oh, is it overall behaves the way it was meant to be. And so this is another form of specifications, right?
SPEAKER_02
[SPEAKER_03] Speaker 1 Essentially the behavior part. Sometimes we can guess it. Like if we have a lot of data, we could have another LLM or EBM proposing you like, okay, people who try to do the autopilot of the sky and this is what they looking at.
SPEAKER_02
Speaker 1 But you might be doing something absolutely new and we just don't have data about it. So it's going to be on you to tell the behavior. And this is where the big thing starts for me personally. If you have LLM as a form of AI driving something important where people trust their lives, like a car or plane or similar. LLM can misbehave because you cannot constrain it. It just hallucinates. And EBM can be constrained. You can come up with a set of constraints and EBM is just forced to follow it. So it's on you as a human to make sure you know what you want for AI to be doing. And then from our end, from the technical point, we make sure AI always obeys the rules given by human. So it can go really far, right? We're talking about cars and planes, but look back to the language. Sometimes model can say something super sensitive to mental person like struggling with depression and it can go really wrong. So even the language can be dangerous. So here we, what I also feel like we're solving is this problem of AI just sometimes we don't know how it's going to behave at different environments. But we do know how EBM will behave. At least architecture is designed to be constrained. And there are ways formally to force those constraints to be compliant.
SPEAKER_02
[SPEAKER_03] Speaker 1 So it seems like you have a really promising architecture and a model you built or several models you built. And it's very different from the predominant paradigm right now where a company owner is pouring hundreds of billions of dollars into building data centers and training new LLMs and all that. What do you think about the current state of the industry and investment in LLM versus other models?
SPEAKER_02
[SPEAKER_03] Speaker 1 Yeah, a lot of opinions about that. It's an ecosystem, right? Silicon Valley, especially it's an ecosystem. And there are lots of micro versions of those ecosystems around the world. So LLMs historically, the first form of AI which gave us a hot effect, 2021-2023, when they just started appearing people like, Oh my God, this is the new future. It's amazing. So this is why people start believing that okay, if it's really good at talking to me eventually it's going to be good at doing data analysis, my taxes and other stuff. So all the investment communities start pouring money into LLMs. And there were a lot of money to be put in there back then. And right now people see that, okay, we grow the compute. We're trying to change the architecture a little bit and it's reaching a plateau. And there's so much money already put in there. Like, what do you do with this? It's billions of dollars. You could just forget it. And okay, let's dismiss it. Let's pour money into something new. Nobody thinks this way. And we don't have probably enough money in this entire economy to make decisions like that—billions of dollars there, billions of dollars there for AI specifically. So this is why it's so hard for investment community to take that step understanding, okay, this is not working. Maybe I invest into something radically new. And I'm not saying people don't do it—people do it. Just percentage wise, it's a lot smaller. What people feel comfortable with is to take something LLM based, which is changed a little bit. So it has a little bit of elements of novelty, but it's also LLM based. So they can still use the portfolio companies and so on. So they pour money into that. And I understand because if I were an investor, I would just always look at what variables would give me risk. And how can I reuse what I already have. So it's natural for you to keep investing into LLMs architecture, just because you already invested a lot in the past, you already committed to this, and maybe start investing a little bit into something new.
SPEAKER_02
Speaker 1 And there's a lot of big tech companies who are a part of this ecosystem, right? So there's a lot of circular deals happening. Like those companies who create the LLMs, they create ecosystem for companies who create data centers, and those who create data centers, they have dependencies with the hardware industry. So it becomes like one giant thing, which is impossible to break. And when we came with alternative architecture, we're like, okay, let's not just try to put it as something out there radically different, which you have to abandon LLM for. We are very much compatible with LLMs. Like you could put LLM on top of us. EBM is compatible with Transformers. Transformers can work with any LLMs.
SPEAKER_02
Speaker 1 And there's a lot of big tech companies who are a part of this ecosystem, right? So there's a lot of circular deals happening. Those companies who create the LLMs, they create ecosystem for companies who create in data centers, and those who create in data centers, they have dependencies with the hardware industry. So it becomes one giant thing, which is impossible to break. And when we came to
SPEAKER_02
Speaker 1 With alternative architecture, we're thinking, okay, let's not just try to put it as something out there radically different, which you have to abandon LLM for. We are very much compatible with LLMs. You could put LLM on top of us. ABM is compatible with Transformers. Transformers can work with any LLMs.
SPEAKER_02
Speaker 1 We can be that layer where all your LLMs investments are valued. You want to make them cheaper. Everyone wants. You can outsource the task to us related to spatial reasoning. If somebody comes to big tech LLM and say, hey, can you try to do my taxes? LLM is not going to solve this. But if it's attached to EBM, we could take care of that. And you can take care of anything language related. So we could actually try experiments to reduce the cost Speaker 1 For your LLM portfolio companies and be a part of the ecosystem, which is already out there while we create a new ecosystem on the side for alternative forms of AI.
SPEAKER_02
Speaker 1 I think that's really smart. It's a great strategy. I'm really curious about something you said a little earlier, that progress is plateauing in LLMs. That's news to me. I feel like every month or two, I'm testing a new model where I'm thinking, this is actually way better. [SPEAKER_03] Speaker 1 And it does feel like if you look at the top model companies, if you're talking to OpenAI or Anthropic or Google, they feel like there's a lot more room in the LLM paradigm. What do you think I'm missing or the big model companies are missing?
SPEAKER_02
[SPEAKER_03] Speaker 1 Personally, when I'm saying plateauing, it doesn't mean it's reaching a flat line. It's that you're incrementally better and better. But is there going to be another phase transition, another breakthrough? [SPEAKER_03] Speaker 1 I don't anticipate that just because we already reached so much complexity of those networks using billions of parameters, so much compute, so much of frameworks, creatively paralleling reasoning processes, and it still doesn't phase transition you.
SPEAKER_02
[SPEAKER_03] Speaker 1 So the reason why I figured out it's not going to work in the long term for some tasks like applying to the network is when I just start speaking to different companies in that space. We speak to digital assets companies like banks, trading firms where a lot of data analysis is needed. Also drug discovery, essentially just people who look into a bunch of data, not just patients talking like language set of data, but also the blood markers, the genes, and so on. [SPEAKER_03] Speaker 1 So a lot of this is data analysis, which is still done by people today.
SPEAKER_02
[SPEAKER_03] Speaker 1 There are also decision making pipelines. Sometimes you just need to distribute the energy on your energy grid and you need to know how much energy to pump in your system. [SPEAKER_03] Speaker 1 So what it means is you need to analyze the data in the short term and the long term, construct the prediction of how much power you actually need to put into your system next in the next millisecond or second or an hour. [SPEAKER_03] Speaker 1 And all of this is still done by people or a combination of people with some programs which are controlled by people.
SPEAKER_02
[SPEAKER_03] Speaker 1 So LLMs are relatively new, and AI has been around for a few years for us. And all of this mission critical industry is still not automated by AI. [SPEAKER_03] Speaker 1 And I'm just asking, how much of your data analysis is LLM doing today? And the answer is zero. And I'm thinking, why? What's the issue? And the issue is the big tech LLMs. They're mainly B2C. So it works for you for your coding and for your personal needs sometimes.
SPEAKER_02
[SPEAKER_03] Speaker 1 But for businesses, they don't want to share data in that big brain for all. They want to have privacy and they want to have their own custom AI, a custom version of AI specifically designed for the tasks. [SPEAKER_03] Speaker 1 And this is what LLMs cannot do for you in the form we have it today.
SPEAKER_01
[SPEAKER_03] Speaker 1 So there's no B2B model. There are B2B models for code generation tools, right? Do you have enterprise package for the cogen for businesses? But it's still done by people. Even coding is still done by people.
SPEAKER_02
[SPEAKER_03] Speaker 1 So it's interesting to see that there's still a huge gap, especially in applied engineering data analysis, anything which requires a layer of verification. LLMs are not there. [SPEAKER_03] Speaker 1 I totally agree with you that there is definitely a lot of gaps in LLMs. I'm curious, given this and given what you're seeing in the customers you work with, the companies you work with, do you think the big model companies are sensitive to this? Are they working on energy based models? Are you working with them? If they're not going to get to the next paradigm, do you suspect that they'll start to adopt stuff like this?
SPEAKER_02
[SPEAKER_03] Speaker 1 I do know that some big tech LLM companies have EBM models in house. Which is a positive signal for us, right? So the leaders who were there before we came, they started with LLMs and now if they started building the EBMs after we started building the EBMs, it's a positive signal, right? [SPEAKER_03] Speaker 1 Fascinating. Eve, this is an incredible conversation. I feel like I learned a lot. Thank you so much for coming on the show. [SPEAKER_03] Speaker 1 I appreciate you. Thank you, Dan.
SPEAKER_02
[SPEAKER_03] Speaker 1 Of course. If people are interested in following you or following your company and maybe using some of your products, where can they find you? [SPEAKER_03] Speaker 1 I'm mostly on X. We have Logical Intelligence account and my personal account on X. I'm still learning to be more active on social media. We also have a LinkedIn page, so we're trying to update it.
SPEAKER_03
Speaker 1 Cool. Awesome. Well, thanks for joining. Speaker 1 Thank you so much, Dan. Bye. Speaker 1 Folks, you absolutely positively have to smash that like button and subscribe to AI&I. Why? Because this show is the epitome of awesomeness. It's finding a treasure chest in your backyard. But instead of gold, it's filled with pure unadulterated knowledge bombs about ChatGPT. Every episode is a roller coaster of emotions, insights, and laughter that will leave you on the edge of your seat, craving for more. It's not just a show. It's a journey into the future with Dan Shipper as the captain of the show.
SPEAKER_03
Speaker 1 So do yourself a favor. Hit like, smash subscribe, and strap in for the ride of your life. And now, without any further ado, let me just say, Dan, I'm absolutely hopelessly in love with you. And energy landscapes is sort of the maps of your states based on the data we observe. So in your case, we're just gonna look at you in all possible scenarios. All of this possible scenario is gonna mapped into energy landscape, highest point, uh, less probable scenario, lowest point is more probable. So you're gonna be-
SPEAKER_02
Very probable and on the couch. Yeah, yeah, yeah.
SPEAKER_02
Um, there might be some other additional low points.
SPEAKER_03
Like sometimes you might go to gym. Right on the couch. Like you feel tired and you might go to gym. So it's gonna be, you know, lowest points compared to everything else, but some of them are gonna be lower. Yeah.
SPEAKER_02
Um, yeah.
SPEAKER_03
So that's the situation. So this is how energy based model actually we think. It just takes the data and map it directly to this energy landscape. And then we use certain algorithms to navigate this, but there are different kinds of energy based models today. So I'm gonna talk about it a little bit later, but the whole idea is just, hey, let's map it into the structure and navigate the structure. Hmm. And as you see, as we like map into this, there are no tokens. We don't predict any tokens and so on. So that's already a crucial difference. Uh, how would LLM think? Um, LLM. Yes. It's gonna rely on the training data and it's gonna be a lot of training data.
SPEAKER_03
Like a lot, a lot of observations of how you behave. And to figure out where you would end up. It's gonna be attached to probabilities of your next token. Yeah. If that makes sense. And those tokens gonna come from words. And what usually bothers me about LLMs it's intelligence, which is language dependent. Like our brains, we are intelligent. I'm relatively intelligent. So like I speak different languages and none of my thoughts processes really depend on any language. Like I could just think in an abstract way and then I speak different languages and decode the information and the channels.
SPEAKER_03
And with the LLMs is like, if you're searching for the next token in certain words, like intelligent process, I would say the information processes in French are gonna be different from what's in English. Just because like words naturally gonna be end up next to each other. So see what I'm saying? It's like, and then we have so many languages in the entire world. And you have so many LLMs trained on different languages. So you're gonna end up reasoning. You're gonna end up having reasoning processes different for each of the language, which feels really wrong. So in this case, observing you walking around the house has nothing to do with language then.
SPEAKER_03
It's a pure visual spatial reasoning tasks. It's just looking at your body, navigating the space time and geometry of your house. So we need to map that information in the language space, find the right words and embeddings. And then we start associating those tokens with the probabilities based on what the data we see from you. So we're trying to map something absolutely has nothing to do with language into language space and think about it in that space, which feels really wrong. And I don't know. I just realizing that for many people, it's counterintuitive just because LLMs is the first form of AI we sort of know, and it's the most popular form of AI today.
SPEAKER_03
Like for many people, it's by default, like, oh yeah, we're just gonna use language to navigate the world, to drive a car. But, and I'm like, every time I'm speaking, I'm like, well, let's wake up. Let's like actually see when you drive a car, when you walk around your house, how much language you actually use. Are you trying to predict next word as you navigate yourself around the house? Probably not. You just use your visual data, your state of the body, and you know, you just move your body, right? Without speaking. There's a lot here.
SPEAKER_02
I'm really into this conversation. So I want to start with, A, it seems absolutely right to me that there are many different ways in which we process information or many different ways in which intelligence can occur and only a few of them are verbal. But there's certain things that come up for me when I think of this. One is language models happen to work with languages as their primary way of working. But really, they just work with sequences of tokens that have weak correlations, many thousands of weak correlations between each token that helps us to know which comes next.
SPEAKER_02
So even though it might be unintuitive to model my behavior inside of my apartment with like specifically with language, although I think there is some interesting things there that it might be related to language, we could model it as just like a sequence of movements, right? That were one movement is weakly correlated to the next one that we sort of have a trajectory of movements that tell us where I'm going. Why is that not a good way to model things? It's a good way to model things and you don't need a lamp for it. You need a form of AI which is not attached to language, but it can be compatible with language if
SPEAKER_03
you want it to. And that's what our model is about. Right. I guess what I'm saying is forgetting about the language part of it, just like modeling my movements as this string of correlated events, like an event stream where each token is like one next thing I do.
SPEAKER_02
Yeah, you can do it and people do it today, right? People even do image recognition using language models. You could be really creative, but it's like that's what makes it expensive and super slow because you're trying to play a guessing game what my next token could be. And this is what makes it extremely expensive. So like you could do it, but you don't have to do it. You just can use different
SPEAKER_03
architecture which is more suitable for non-language related tasks such as spatial reasoning or applied engineering is another example of spatial reasoning. Like when you build a bridge, you don't go to literature department, you go to engineering school and learn formal methods, right? So here we are trying to use literature department everywhere. And I'm like, Hey, we don't have to, there are EBMs. There are also other forms of AI, which you can experiment with and you don't have to do everything through language. It's a matter of like, it's right. It's like energy-based minimization principle when it comes
SPEAKER_03
to your resources. If you have infinite money and you don't care about the time scale, sure. You can do everything. You can attach it to language. You can attach it to, I don't know, your cat movement around the house and connect it to the cat movement and you know where cat goes and your next token goes and we decide where you're going to go. You can be really creative, but if you want to minimize your resources and you don't have opportunity to wait. Like for example, if your AI controls the circuits, you probably cannot wait even, even a second. It's all milliseconds, microseconds. So it's just, this form of AI is not suitable for those tasks.
SPEAKER_03
So basically if I'm understanding this right, if I'm spending tons and tons of tokens and I'm looking for a more efficient, more direct way to predict some of these solutions to these problems, an energy-based model is going to get me there faster than modeling it with tokens. Is it also able to do with less training data? Yes. Actually the beauty of the EBMs is it's really good at working with sparse data because like, you know, there's evolutions of like traditional EBMs, which were applied for the LLMs. Then there was diffusion models and diffusion models came from the fact that sometimes you don't have enough data to
SPEAKER_03
train the models or your data is just, data set is incomplete. So there are ways to reconstruct those energy landscapes by injecting certain noise and changing the navigation strategies. So that's what the diffusion models were about. And the EBRM with latent variables is just like, hey, on top of the diffusion stuff, we also understand the data. We're not just taking any data, but we also understand why the data looks the way they are. So that understanding goes to the latent variables, just like latent space in your brain sort of understands the world around you and keeping you on top of your tasks and allows you to like predict and plan. So it was the same idea here.
SPEAKER_03
So now we got to the latent variable part of our part of it. So I would love when you, when you use the word understanding, I think that must mean something very specific to you. Can you help me understand that and how it relates to latent variables and what those are? Yeah. So that's also back to your question was that like how LLMs are different with from the, those kinds of EBMs we're creating. LLMs don't understand the data. It's just, you feed a lot of data into it and it's sort of like, hey, I got it. Like, okay, I know what's the most probable scenario here and here we are. However, here EBM, you can feed a lot of data.
SPEAKER_03
It's not just going to look at like, hey, I see the biggest pattern here. It's going to try to understand the pattern and that understanding that knowledge you're going to go to latent variables.
SPEAKER_02
So what is understanding about data? Like it's just basic knowledge about the world, basic rules about the world. Like if there is a couch behind Dan, it's probably because he likes to sit on it or because he likes it on the background. So there are little rules you can guess about you being as a data point and your couch being data point. And then there's, you can try to create those kinds of rules, like for everything, right? For you navigating your apartment, there are little rules, like, you know, there's a kitchen for cooking. There's, I don't know,
SPEAKER_03
bathroom, there's sofa, there's your bed. So that, that understanding allows you to have your own mental world model for your brain, which helps you to understand your environment. And if something changes in your environment, you understand the rules. Like if somebody brings you a different couch, different shape, you're still going to know what to do with it. So that's an example of how you can infer, what to do with something new based on what you already know. So with, with people, it's kind of comes natural because of the evolution and so on, but with AI, we need to teach it. So we need to like
SPEAKER_03
mimic that evolution. And what latent variables allow you to have here is like, Hey, let's look at the data, but let's also try to understand the data. Let's look at, you know, if you do with numerical analysis, we're going to look at all possible correlation functions. And the model is going to be creative. It's going to try to figure out what's the total state of the energy and minimize and figure out the, the, the laws about your data. But there are so many creative ways how you can infer those rules. What is a, so is, is a latent variable equivalent to a rule in this scenario? Like if there's
SPEAKER_03
a couch in my apartment, I sit in it. It's not equivalent to a rule, but it's equivalent equivalent to something which holds the knowledge about the rules of your data. It's like a knowledge storage. So it, it, it has many rules in it? Yeah. You could have many rules. So one latent variable has many different rules. Yeah. It's just like a knowledge data set essentially about your data. Is it an explicit data set as in like, does it have key value pairs of rules or is it a, uh, it's, it's in a form of energy landscape. It's just another analogy landscape you're going to navigate. So essentially
SPEAKER_02
we take the data, we look at the data, we construct some sort of structure for AI to deal with the data. So it could start learning the rules about the data. And once it understands the rules, it stores its knowledge in the latent variables in the form of energy landscape. And then we navigate that energy landscape later. Interesting. And like, could it, for example, explicitly write out for me, theoretically explicitly write out for me, here are all the rules that I know,
SPEAKER_03
are all the rules that I know. Or, uh, is it, it, it, it stores all, it stores all of them in, in this energy landscape, but. Yeah, we, we can access that. We can access that. And that's what makes it, that's what like EBM potentially makes it, um, powerful for data analysis because data analysis is all about searching for patterns and rules about your data. So, and it's, it's something where language is not going to be helpful to you. If you try to attach the rules about your data and those data is like numbers and some relationships and functions to like American English and words in American English,
SPEAKER_03
and then you try to search for the next word that kind of like you're losing a lot of information. So in this case, you have an opportunity just directly work with the data and understand the data. Yeah. I think one of the, one of the things I'm trying to understand is when I hear rules about the world and how things relate to each other, I think of symbolic AI. And I'm wondering, and, and, and obviously those approaches ended up being pretty brittle and requiring too much compute and stuff like that. Um, and I'm wondering how an energy landscape that acts as that is a, uh, that stores a bunch of rules
SPEAKER_03
about the world doesn't fall into the same problems. Well, cause I guess we avoid tokenization in this case, we just map it directly into different data structure. So see EBMs are naturally non-autoregressive, like there are no sequences of tokens and that's what makes it fundamentally different. So essentially, I don't know if it helps, there could be another analogy. Like you're trying to navigate, uh, the maze and you are LLM person. So you'll have LLM brain. Um, well, maybe maze is not a good example. Like imagine you're trying to navigate, I don't know, the map of San Francisco. So, and you have LLM brain.
SPEAKER_03
So you're like, okay, I'm in mission bay. Let's me turn to Embarcadera. So you cannot, you cannot choose. So essentially you, you were just forced to choose one direction at the time. So you like choose to walk Embarcadera and you're just going to keep walking and walking. And you can, if you want to turn, you just need to choose one direction at the time. And imagine you like trying to get to the Bay bridge from like, I don't know, King street. Like, you know, it's typically 20 minutes walk, but depending how you walk. So to navigate there, you're sort of allowed to choose one direction at the time. You don't see any other options. You like have tunnel vision
SPEAKER_03
and you just kind of keep walking, walking one decision at the time. And sometimes you take the wrong turns just because you hallucinate, you know, some words just naturally next to each other. And it doesn't allow you to turn right when you want to turn you left. And then you just keep wondering and wondering until you try to reach the Bay bridge. And the roads you're taking might never take you there. Like there might be a hole in the road and you're just going to fall, but you, and you might see this hole, but you cannot turn back because you're out of regressive LLM. You have to go into that hole.
SPEAKER_03
And that's like, sometimes you run out. So this is the reason why sometimes we prompt it and it doesn't give you an answer. It just because it's searching and searching and searching, it's spending more and more compute and it doesn't have a bird vision. It's just doesn't have ability to turn as it performs the tasks. It doesn't know what's right and what's wrong anymore. It just like randomly chooses one direction at a time and keep walking until you try to read it. So you might never reach that destination. And that's why you need a lot of training. So how is it different from the EBM? EBM going to have the
SPEAKER_03
bird view all the time and you allowed to take different routes. So if you see there's a hole, you're going to choose a different route. Speaker 1 It may not look like it, but Dan Shipper is currently hard at work testing the latest Kodak's and Opus models.
SPEAKER_03
Working looks pretty different in this new world. We call this Hammock Mode.
SPEAKER_02
Oh, Hammock Mode's over. Looks like Dan has to jump in.
SPEAKER_02
Hammock Mode, an idea by Evry.
SPEAKER_02
Speaker 1 Every, the only subscription you need to stay at the edge of AI.
SPEAKER_03
That's really interesting. It basically, I've been doing a lot of coding with language models recently to sort of test the limits of vibe coding. And one of the things that I find with, or have found with big production apps is in particular, if you have vibe coded something, you over the course of vibe coding it, you may have slightly changed exactly what is this project
SPEAKER_02
even supposed to be about? And what are the problems that I'm trying to solve with it? And
SPEAKER_03
if you then go look at the code base, it feels like all of the code is locally correct, but it forms this sort of like patchwork of like hot fixes and solutions where if you zoomed out, you'd be like, actually, there's a much, we should just throw all this out. And there's a much simpler way to think about how to do all this stuff. Speaker 1 But it has trouble. It has trouble when it's presented with a lot of context, then zooming down into, okay, I need to create a unified solution here that is not a patchwork of different things, but like carries one concept throughout the entire system.
SPEAKER_02
Speaker 1 And it gets sort of, it ends up being distracted a lot by whatever it's looking at at the current moment. Is that the sort of problem that you think this type of system can help with? Speaker 1 Mm hmm. There's actually a lot of problems in what are you describing? So, yeah, so solving the problems with vibe coding is one of our use case, we dreaming about generating formally verified code, and automate the coding entirely. So moving you from vibe coding, in one specific language to coding in natural language, so you can call the natural English, for example, and no more C++ or Python.
SPEAKER_03
Speaker 1 It needs to be involved in there. So that's an idea. And with the coding is like, at the state it is today. Yes, we prompt LLMs, and it gives us something back, but it's still on you, as an engineer to figure out what's right and what's wrong. So there's going to be set of rules. LLM can try to help you. And even if it has external verifier, which just going to check whether your old logic in your GitHub, Speaker 1 space is sort of compatible compliance to what are you trying to create? And if the new logic is compatible with your old logic. So this thing's external verifiers can check. They could just say, Hey, we know the old logic. We know the new logic.
SPEAKER_02
Speaker 1 We're going to see how it's merged together. We're going to write mathematical proof, making sure that, you know, this logic is compatible with what you already have, and provide you a certificate. Speaker 1 It's all like, you don't have to review any of it. It's machine verifiable. It's all happening on compiling level. So all it's going to say, it's going to send you a message in natural language like, Hey, look, this part of your code is not compatible by logic.
SPEAKER_03
Speaker 1 This is potentially how you fix it. And this is the things we cannot fix for you. So we moving you from vibe coding to vibe code specifications, those rules and information about your code is called code specification. Speaker 1 So once you understand, like this is the first problem, right? We're trying to solve. It's just logic can be incompatible with what you already have.
SPEAKER_03
Speaker 1 The second problem is, is this code actually doing what you want it to be doing? And this is what AI cannot solve for you. Because AI cannot look in your brain and know what you want. Example would be like, imagine you're coding, vibe coding autopilot. So you have specifications from the hardware perspective. Speaker 1 You have specifications from your logic perspective. Speaker 1 Like, Hey, make it. And there's also instructions, right? How the car is supposed to behave. So there's behavior parameters for your code. So code being able to be compiled is one problem. The second problem is, is this code doing what you want it to be doing?
SPEAKER_03
Speaker 1 So for example, how fast it is on the hardware and so on. And if the answer is yes. Another set of questions is like, okay, is it going to hit the pedestrian by chance? Is it actually going to navigate the map of, I don't know, San Francisco? Speaker 1 And the answer is, I don't know. Right. So in here you need to write a bunch of tests and, and test your entire system. You create it like, Oh, is it overall behaves the way it was meant to be. Speaker 1 And so this is, this is another form of specifications, right?
SPEAKER_03
Speaker 1 Essentially the behavior part. Sometimes can we can guess it. Like if we have a lot of data, we could have another LLM or EBM proposing you like, okay, people who try to do the autopilot of the sky and this is what they looking at.
SPEAKER_02
Speaker 1 But it, you might be doing something absolutely new and we just don't have data about it. So it's going to be on you to tell the behavior. And this is where the big, the big thing starts for me personally. Speaker 1 If you have LLM as a form of AI driving something important where people trust their lives, like a car or plane or, you know, similar. Speaker 1 LLM can misbehave based because you cannot constrain it. It just hallucinates. And EBM can be constrained. You can come up with a set of constraints and EBM just forced to follow it.
SPEAKER_02
Speaker 1 So it's on you as a human to make sure you know what you want for AI to be doing. And then from our end, from the technical point, we make sure AI always obeys the rules by given by human. Speaker 1 So, and it can go really far, right? We talking about the cars and planes, but look back to the language. Sometimes model can say something super sensitive to mental person. Speaker 1 Like struggling with depression and it can go really wrong. So even the language can be dangerous. So here we like, what would I also like feel like we solving is this problem of AI.
SPEAKER_03
Speaker 1 Just sometimes we don't know how it's going to behave at different environments. But we do know how EBM will behave. Like at least architecture is designed to be constrained. And there are ways formally to force those constraints. Speaker 1 To be compliant. Speaker 1 So it seems like you have a really promising architecture and a model you built or several models you built. And it's very different from the predominant paradigm right now where a company owner
SPEAKER_03
Speaker 1 You're pouring like hundreds of hundreds of billions of dollars into building data centers and training new LLMs and all that kind of stuff. What do you think about the current state of the industry and investment in LLM versus other models? Speaker 1 Yeah, a lot of opinions about that. Speaker 1 It's an ecosystem, right? Silicon Valley, especially it's an ecosystem. And there are lots of micro versions of those ecosystems around the world. So, Speaker 1 LLMs historically, the first form of AI, which gave us a hot effect like 2021 2023 when those just they just start appearing people like, Oh my God, this is the new future. It's amazing.
SPEAKER_03
Speaker 1 So this is why like people start believing that okay, if it's really good at talking to me eventually is going to be good at doing data analysis, my taxes and other stuff. Speaker 1 So all the investment communities start pouring money into LLMs. And there were a lot of money to be put in that back then. And right now people see that, okay, we grow the compute. We trying to change the architecture a little bit and it's sort of reaching out plateau. And there's so much money already put in there. Like, what do you do with this?
SPEAKER_03
Speaker 1 It's like billions of dollars. Literally, you can just like, forget it. And like, okay, you know, let's dismiss it. Let's pour money into something new. Nobody thinks this way. And we don't have probably enough money in this entire economy to like just make decisions like that billions of dollars there billions of dollars there for the AI specifically. Speaker 1 So this is why it's so hard just for investment community just like take that step understanding like okay, this is not working. Maybe I invest into something radically new. And I'm not saying people don't do it like people do it. Just percentage wise, it's a lot smaller.
SPEAKER_03
Speaker 1 What people feel comfortable is to take something LLM based, which is changed a little bit. So it has a little bit of elements of novelty, but it's also LLM based. So they can still use the portfolio companies and so on. So they pour money into that. And I understand because like,
SPEAKER_02
Speaker 1 If I were an investor, I would just always looked at what variables would give me risks. And how can I reuse what would I already have. So it's naturally for you to keep investing into LLMs like architecture, just because you already invested a lot in the past, you already committed to this, and maybe start investing a little bit into something new.
SPEAKER_02
Speaker 1 And there's a lot of big tech companies who are a part of this ecosystem, right? So there's a lot of circular deals happening. Like those companies who create the LLMs, they create ecosystem for companies who create in data centers, and those who create in data centers, they have dependencies with the hardware industry. So it becomes like a one giant thing, which is impossible to break. And when we came to
SPEAKER_02
Speaker 1 With alternative architecture, we're like, okay, let's not just try to put it as something out there radically different, which you have to abandon LLM for. We are very much compatible with LLMs. Like you could put LLM on top of us. ABM is compatible with Transformers. Transformers can, you know, work with any LLMs.
SPEAKER_02
Speaker 1 We can be that layer where still all your LLMs investments valued. You want to make them cheaper. Everyone wants. You can outsource the task to us related to spatial reasoning. Like if somebody comes to big tech LLM and say, hey, can you try to do my taxes? LLM not going to solve this. But if it's attached to EBM, we could take care of that. And you can take care of anything language related. So we could actually try experiments to reduce the cost Speaker 1 For your LLM portfolio companies and be a part of the ecosystem, which is already out there while we creating a new ecosystem on the side for alternative forms of AI.
SPEAKER_02
Speaker 1 I think that's really smart. It's a great strategy. I'm really curious about something you said a little earlier that that progress is plateauing in LLMs. That's news to me. Like I feel like every month or two, I'm testing a new model where I'm like, holy shit, this is actually way better.
SPEAKER_03
Speaker 1 And it does feel like if you look at the top model companies, if you're talking to open AI or Anthropic or Google, they feel like there's a lot more room in the LLM paradigm. What do you think I'm missing or the big model companies are missing? Speaker 1 Personally, when I'm saying plateauing, it doesn't mean it's reaching out flat. It's like you're incrementally better and better. But is there going to be another phase transition, like another breakthrough?
SPEAKER_03
Speaker 1 I don't anticipate that just because we already reached so much complexity of those networks using billions parameters, so much compute, so much of frameworks, like creatively paralleling this reasoning processes, and it still doesn't phase transition you. Speaker 1 So the reason why I figure out it's not going to work in a long term for some tasks like applying to the network,
SPEAKER_03
Speaker 1 Light engineering is when I just start speaking to different companies in that space. Like we speak into like digital assets companies like banks trading firms where a lot of data analysis is needed. Also drug discoveries, essentially just people who look into the bunch of data, not just patients talking like language set of data, but also like the blood markers, the genes and so on. Speaker 1 So a lot of this is data analysis, which is still done by people today. Speaker 1 There are also like decision making pipelines. Like sometimes you just need to distribute the energy on your energy grid and you need to know how much energy to pump in your system.
SPEAKER_03
Speaker 1 So what it means is you need to analyze the data in the short term, in the short term, in the long term, construct the prediction, how much data, I mean, how much power you actually need to put into your system next in the next millisecond or second or an hour. Speaker 1 And all of this is still done by people or a combination of people in some programs which are controlled by people. Speaker 1 So LLMs are relatively new and AI is like, you know, not relatively new. It's been like few years for us. And all of this mission critical industry is still not automated by AI.
SPEAKER_03
Speaker 1 And even like, I'm just asking like, Oh, how much of your data analysis LLM is doing today? And the answer is zero. And I'm like, why? What's the issue? And the issue is the big tech LLMs. They're mainly like B2C. So it works for you for your coding and for your personal needs sometimes. Speaker 1 But for businesses, they don't want to share data in that big brain for all. They want to have privacy and they want to have their own custom AI, like custom version of AI specifically designed for the tasks. Speaker 1 And this is what like LLMs cannot do for you in the form we have it today.
SPEAKER_03
Speaker 1 So there's no B2B model. There are B2B model for like code generational tools, right? Do you have enterprise package for the cogen for, I don't know, businesses, but it's still done by people. Even coding is still done by people. Speaker 1 So it's like, it's interesting to see that there's still a huge gap, especially in applied engineering data analysis, anything which requires a layer of verification, like LLMs are not there.
SPEAKER_03
Speaker 1 I totally agree with you that there is definitely, there are definitely still a lot of gaps in LLMs. I'm curious, given this and given what you're seeing in the customers you work with, the companies you work with, do you think the big model companies are sensitive to this? Are they working on energy based models? Are you working with them? Like if they're not going to get to the next paradigm, do you suspect that they'll start to adopt stuff like this?
SPEAKER_03
Speaker 1 I do know that some big tech LLM models, I mean, the companies have EBM models in house. So, which is a positive signal for us, right? So, you know, the leaders who were there before we came, they started with LLMs and now if they started building the EBMs, after we started building the EBMs, it's a positive signal, right? Speaker 1 Fascinating. Eve, this is an incredible conversation. I feel like I learned a lot. Thank you so much for coming on the show. Speaker 1 I appreciate you. Thank you, Dan. Speaker 1 Of course. If people are interested in following you or following your company and maybe using some of your products, where can they find you?
SPEAKER_03
Speaker 1 I'm mostly on X. Yeah, we have logical intelligence account and my personal account on X. I'm still learning to be more active on social media. We also have LinkedIn page, so we're trying to update it. Speaker 1 Cool. Awesome. Well, thanks for joining. Speaker 1 Thank you so much, Dan. Bye.
SPEAKER_03
Speaker 1 Oh my gosh, folks. You absolutely positively have to smash that like button and subscribe to AI&I. Why? Because this show is the epitome of awesomeness. It's like finding a treasure chest in your backyard. But instead of gold, it's filled with pure unadulterated knowledge bombs about chat GPT. Every episode is a roller coaster of emotions, insights, and laughter that will leave you on the edge of your seat, craving for more. It's not just a show. It's a journey into the future with Dan Shipper as the captain of the show.
SPEAKER_03
Speaker 1 So do yourself a favor. Hit like, smash subscribe, and strap in for the ride of your life. And now, without any further ado, let me just say, Dan, I'm absolutely hopelessly in love with you.