Think You Can Build a Game with AI? Think Again! - Danielle An & David Hoe, Meta

Transcript

Hello, thank you for joining us before lunch especially. I appreciate it. I'm David from Meta. [SPEAKER_00] I'm Danielle. I'm a principal engineer at Meta. And we have a little set of slides here we thought would make a different experience today. So instead of non-interactive slides, you get some interactive slides. So if anyone's interested, you can just scan this QR code or try and make it bigger for you. And if you get fidgety, have a go and see what this does along the way. This is linked up to, by the way, so instead of slides, each tap will reveal the slide content. So depending on how chaotic you want things to be, you have control. Everyone got that? I'm going to close that down. And over to Danielle. Yeah, so while you guys are getting the hang of this, it seems to be working. Multiple people are controlling it. There's a voting system behind the scenes that will determine who wins. But if you try to, yes. It's a little quest-giving game. But before we go any further, the reason we're doing this is that the two of us are leading some of the AI-driven gaming creation work at Meta. We've been doing that for over a year now. And a lot of what we do on a daily basis is using AI to help generate games for the casual creator, pro creator, and everybody alike. So given that that's our full-time job, we figured that we will take you on a little bit of a journey on the games that David made in just the last 12 hours. So now we're going to get started. So because there's lunch and the previous talks are a little bit delayed, we appreciate you being here, as David was saying. But we should also be honest about why you're here. And we want to tell you what kind of value we might bring or not bring. So we're imagining if you come to this talk, you're people that are very interested in games. Maybe you play a lot or at least you're somewhat interested in how games are made. And even better, if you wanted to make them in the future or if you made them in the past and you're wondering how AI is going to change that. So this is the kind of talk for you. But if none of that is interesting to you, at least you can just have fun, play the game before lunch, and work up your appetite. But now we're going to get going on to the real part. In 2025, a lot of you probably have either built something like a little game, a platformer game, or Tetris or something with an LLM. If you have not, maybe your kids, your moms, your friends have built them. Very likely chances are it's very likely. And initially, it's very impressive because within a couple of prompts, you're getting, oh, here is a game. I can recognize this. And if you're lucky, it works out of the box, but not guaranteed. But the problem is that as time goes on, you would all see that the novelty wears off. And a lot of what you prompt is the same. If everybody says, I want a little platformer game that looks like Mario, overall, they all kind of look similar if they work at all. So after a while, what's next? This is cool. So that's, yes, we're all moving a lot here. So if people want to see what the text actually is, it's good to just not move for a while. But how many people are here? Actually, I'm not sure how many people are here, but a lot of people are controlling, so it's hard to stabilize, which is part of the game. But the point here is that everybody can build a game over the weekend, but what is the next thing? And what does this mean, really, to the industry? And in our opinion, what this means is that a lot of people previously who wanted to build games but were gated by skill sets. For example, I'm a coder, but I'm not an artist. I cannot do 3D modeling. I cannot draw. I cannot get art. But now you can just use DALL-E or Midjourney or whatever else to get yourself unblocked. Same thing with artists who previously really wanted to build games, but maybe they don't have the coding skills, and so they have to wait for engineers or they just feel like that's not possible. For a set of people that really want to utilize this moment to build more games, the barriers are removed, but that doesn't mean everybody can make a good game. What will separate a good game versus a bad game is still a lot of the typical stuff, and here we're going to share with you some of the tips that we've learned in the last year working on AI game building, which is how do you stand out? I hope so far you are not too bored. How do you stand out? There's a few things. One is usually aesthetics. After you prompt a few games, you're not satisfied with a little box that's purple next to a box that's green that shows a platform. There's a lot of art to that. There's also a lot of cohesion to the game you make between the UI and the stories and art. Does it feel like one entity? Does it feel like it lives in one universe? And then there's lots of play tests and real people with real feedback will tell you what is good. But I'm just going to directly go to a video that maybe David can comment on. What is the point of this video? [SPEAKER_01] Yeah, so this is a video from one of our directors that we work with, Dale. And he wanted to demonstrate here an example workflow that you could try out, which is really about using key art as an anchor for the LLM. So just like game development, you might have a stage where you're iterating and you come across a concept that you really like. So the same thing with the models. You can use just a single key art image for game for inspiration. In this case, you can see the key art was a lovely bear example. And the difference here is that you go from being able to, it basically anchors you on having an image that you can use for the art style that you can then filter down to the assets that you can see here before. But also, it can also help anchor on what the gameplay could be. which is really about using KeyArt as an anchor for the LLM model. So just like game development, you might have a stage where you're iterating and you come across a concept that you really like. So the same thing with the models. You can use just a single KeyArt image for game inspiration. In this case, you can see the KeyArt was a lovely bear example. And the difference here is that you go from being able to anchor you on having an image that you can use for the art style that you can then filter down to the assets that you can see here before. But also, it can help anchor on what the gameplay could be. So it actually holds a lot of information and it's a very simple way to get started and allow for the LLM to have some cohesion throughout the sessions as well. So, yeah, that's just demonstrating that. [SPEAKER_00] So this hopefully shows you the difference between something you can prompt or your kids can prompt versus a professionally made game. [SPEAKER_00] And the distance is shortened by a lot because what David is saying, all the AI tools that can help you with that. [SPEAKER_00] But another, once you get to the basic level of, okay, this looks like a decent game. [SPEAKER_00] One of the most important things is what will make your game stand out if every game looks polished? [SPEAKER_00] What makes your game the best, not just better? And what makes it the best is the same as everybody now in the AI age would say is the taste. What it really means to the games is that it's not that you delivered a game, but it's that you have the feel that this is a game humans would and which subset of the humans would and why they would have fun with that. What are they pursuing behind the scenes with these games? And then once you reach that level, you're, ah, great. Now I can make a game. Now the problem is not quite because now there's introduction of new technology, runtime LLMs. What that means is that while you are playing the game, there is a living entity, the runtime LLM, that is modifying, changing the game, directing the game in some way. Here is another example of that. Would you like to comment on that? Our team built this, again, just over a couple of days. And this is a multiplayer game where each of the NPCs here is entirely driven by LLM. So this is the part where the humans are setting up a competitive game. It's four players playing against each other. And by specifying what the NPC should do, giving it a personality, maybe it's a thief, maybe it's very honorable, maybe it's fast, whatever personality it has. Now the game is about to begin. And you will see these runtime LLMs independently making decisions, achieving the goal of this game, which is to get as many cubes as possible. But as you watch, you will see some of the NPCs will make decisions to steal other NPCs' cubes or block other NPCs, kick other NPCs. These are entirely not scripted. These are runtime LLM driven decision making. So it just adds a lot of spice, a lot of dynamicness to the games. Every game is unique. Every game is not repeatable. And so that is a new form before the introduction of AI in the last, let's say, 18 months, which is not possible. So now we're experimenting a lot with this genre of games where you can see these NPCs starting to fight with their personalities. You can obviously design the game to be more spicy or more friendly, co-op or competitive. And again, the point of these games is that they were previously not in the industry. It was just not possible to make them. But now the inference time is fast enough. The models are cheap enough for us to actually build these kind of games. Again, built over a couple of days. So next, we're going to share some learnings from us driving these teams in the last 12 months. Feels 12 years. Directing teams across disciplines and what we've learned using AI tools and how does it change the game building experience. Briefly, the first thing it changes is in the past, building games is very expensive because it's a linear process. From one department to another, first you design and then you go to art. Then you go through modeling potentially. You go through animations. You go through coding. But all of this is a linear waterfall model. And what that means is that once you commit to making a game, it's very hard to revisit the decisions upstream. They're very costly. But nowadays, because of AI, we have achieved teams that are working in parallel and updates and iterations on the games are achieved in hours and days. As opposed to months. What this achieves is just that games are more fun because you have more play tests. You have more time to iterate on the game idea as opposed to just linearly executing. And this is a game changer. And the second thing that we've learned on top of the taste is just with LLMs, with runtime decision making, there's a game master that can help you make the game more personal, more interesting for you. For example, for myself, I'm coordination challenged. So when I play games with my friends, if it's co-op, I'm always the one that is tanking the team. And in the past, it just sometimes stops me because I feel I'm just dragging the team behind. But if LLMs can adjust that for me, allow me to still play with my team and have fun, that is a big win that previously was not possible. Another learning is, as we previously said, runtime LLMs is going to be a game changer. But just any piece of technology, the technology itself does not make the product better. We are, just as a lot of the teams, still at early phases exploring how do you use runtime LLMs? We imagine a lot of these things, Star Trek, right? You say, imagine in front of me, there's a forest and now I'm hunting bears and it just suddenly happens. We do see that eventually becoming reality, but it's not currently the reality today. So today, how do you use runtime LLMs to drive your game to be differentiating against other games? It's still something we're exploring and a lot of people are very interested in this area. But in general, there's lots of challenges. I think everybody, whether you're in the game industry or other industry, what you do feel is that every day, AI is changing what you are doing from the tools, from your code base, to what good looks. And just keeping up with that has been very hard, very challenging, but also very interesting. If that's the kind of thing that you, that's going to take you a really long way. It's refusing to talk to the NPC now. And the last bit I'll talk about this, technology driven, is now you imagine we work at Meta. But in general, there's lots of challenges. I think everybody, whether you're in the game industry or other industry, what you do feel is that every day, AI is changing what you are doing from the tools, to your code base, to what good looks like. And just keeping up with that has been very hard, very challenging, but also very interesting. If that's the kind of thing that you like, that's going to take you a really long way. It's refusing to talk to the NPC now. And the last bit I'll talk about this, technology driven, is now you imagine we work at Meta. Meta cares about being a platform of millions and millions of pieces of gaming content that will come online, especially now that everybody can create. One of the main challenges is just that through all of the agentic systems from the front end, where the user is prompting potentially to runtime the LM is making decisions like the game we showed before, to the platform. Serving content, ranking content, delivering content. Through the whole platform there's agents and agentic systems at play. So that just means undeterminism throughout the whole stack. [SPEAKER_00] Which is something that as engineers, especially at scale, we are struggling a lot with in general, right? [SPEAKER_00] Because we're used to thinking that you write code, you write tests, stability, debugging against, is set known, a known set of code base is what we call stability and scalability. [SPEAKER_00] But now in these days, if your model is changing, model upgrades, if your prompt is changing, a lot of things can entirely throw off your system. [SPEAKER_00] So how do you even debug that? [SPEAKER_00] How do you engage with systems like that is still a challenge. [SPEAKER_00] But since we have only two minutes left and it's about lunch time, let's say something like that is, that inspires David and I on a daily basis is that this just means that for the whole gaming as an industry, is being transformed, being transformed by AI and we're at day zero. [SPEAKER_00] So for those of you who have read or seen Ready Player One, this is the moment where we think we're at the beginning, where that oasis can actually be a reality. [SPEAKER_00] So if you want to be part of that, I do really hope you guys navigate to her because there's something very cool here, you can navigate to her. [SPEAKER_00] Oh, please. [SPEAKER_00] It's not happening. [SPEAKER_00] Come on, please. [SPEAKER_00] Can we go to her? Ah, yes. So I personally love Labubu. So this is one of those personalization moments where with AI, you can see any game towards anything that you want. And for me, building this moment just means that you can create a universe of Labubu. So you can be any Labubu you want. You can collect any Labubu you want. And last but not least, a little bit of takeaway before lunch is, whoever it is you are, maybe you've never built a game. So then the task here for the takeaway for you is the weekend is coming. You can just use any of your favorite models, maybe Gemini, Manus, whatever it is. Just try to prompt a very basic game like Tetris or Infinite Runner, something that you play, and just see what it does. It's going to fail. It's going to bring you delight when it surprises you and it works really well. Or maybe you are the people that already went through that phase and that's why you're in this talk. What you're thinking is, what is the next phase? The next thing for you is probably something we mentioned prior is to upgrade your game with art, with UI, with surprises. So it doesn't feel like it's something that is prompted out of the gate. So you can up level there. But if you are beyond that as well, then if you're an engineer that cares about scalability, then think about tokens. How do you make sure creators, players, and platform with token economy are going to be profitable? And content safety is one of the bigger challenges. How do you make sure with runtime online, maybe potentially generating images and content, that the content is actually safe for your audience. There's a lot of open challenges there. Wherever it is you are, I mean negative time. So thank you very much for being here from me and David. Thank you for participating and working with the NPCs. Hopefully you had a little fun and go to lunch. Thanks. You can obviously design the game to be more spicy or more friendly, co-op or competitive. And again, the point of these games is that they were previously not in the industry. It was just not possible to make them. But now the inference time is fast enough. The models are cheap enough for us to actually build these kind of games. Again, build over a couple of days. So next, we're going to share some learnings from us driving these teams in the last 12 years. 12 months. Feels like 12 years. Directing teams across disciplines and what we've learned using AI tools and how does it change the game building experience. Briefly, the first thing it changes is in the past, building games are very expensive because it's a linear kind of process. From one department to another, first you design and then you go to like art. Then you go through modeling potentially. You go through animations. You go through coding. But all of this is kind of like a linear waterfall model. And what that means is that once you commit to making a game, it's very hard to revisit the decisions upstream. They're very, very costly. But nowadays, because of AI, we have achieved teams that are working in parallel and updates and iterations on the games are achieved in hours, days, and days. As opposed to months. What this achieves is just that games are more fun because you have more play tests. You have more time to iterate on the game idea as opposed to just linearly executing. And this is a game changer. And the second thing that we've learned on top of the taste is just with LMS, with runtime decision making, there's a game master that can help you make the game more personal, more interesting for you. For example, for myself, I'm coordination challenged. So when I play games with my friends, I'm always, if it's co-op, I'm always the one that is like tanking the team. And in the past, it just sometimes stops me because I feel like I'm just dragging the team behind. But if LMS can adjust that for me, allow me to still play with my team and have fun, that is big win that previously was not possible. Another learning is, as we previously said, runtime online is going to be a game changer. But just like any piece of technology, the technology itself does not make the product better. We are, just like a lot of the teams, still at early phases exploring how do you use runtime online? We imagine a lot of these kind of things like Star Trek, right? You say, imagine in front of me, there's a forest and now I'm hunting bears and it just suddenly happens. We do see that eventually becoming reality, but it's not currently the reality today. So today, how do you use runtime on LMS to drive your game to be differentiating against other games? It's still something we're exploring and a lot of people are very interested in this area. But in general, there's lots of challenges. I think everybody, whatever you're in the game industry or other industry, what you do feel is that every day, AI is changing what you are doing from the tools, from to your code base, to what good looks like. And just keeping up with that has been very hard, very challenging, but also very interesting. If that's the kind of thing that you like, that's going to take you a really long way. It's refusing to talk to the NPC now. And the last bit I'll talk about this, kind of technology driven, is now you imagine we work at Meta. Meta cares about being a platform of millions and millions of pieces of gaming content that will come online, especially now that everybody can create. One of the main challenges is just that through all of the agentic systems from the front end, where the user is prompting potentially to runtime the LM is making decision like the game we showed before, to the platform. Serving content, ranking content, delivering content. Through the whole platform there's agents and agentic systems at play. So that just means undeterminism throughout the whole stack. Which is something that as engineers, especially at scale, we are struggling a lot with in general, right? Because we're used to thinking that you write code, you write tests, stability, debugging against, is set known, a known set of code base is what we call stability and scalability. But now in these days, if your model is changing, model upgrades, if your prompt is changing, a lot of things can entirely throw off your system. So how do you even debug that? How do you engage with systems like that is still a challenge. But since we have only two minutes left and it's about lunch time, let's say something like that is, that inspires David and I on a daily basis is that this just means that for the whole gaming as an industry, is being transformed, being transformed by AI and we're at day zero. So for those of you who have read or seen Ready Player One, this is the moment where we think we're at the beginning, where that oasis can actually be a reality. So if you want to be part of that, you're going to, I do really hope you guys navigate to her because there's something very cool here, you can navigate to her. Oh, please. It's not happening. Come on, please. Can we go to her? Ah, yes. So I personally love Labubu. So this is one of those personalization moments where with AI, you can see any game towards anything that you want. And for me, building this moment just means that you can create a universe of Labubu. So you can be any Labubu you want. You can collect any Labubu you want. And last but not least, a little bit of takeaway before lunch is, whoever it is you are, maybe you've never built a game. So then the task here for the takeaway for you is the weekend is coming. You can just use any of your favorite models, maybe Gemini, Manus, whatever it is. Just try to prompt a very basic game like Tetris or Infinite Runner, something that you play, and just see what it does. It's going to fail. It's going to bring you delight when it surprises you and it works really well. Or maybe you are the people that already went through that phase and that's why you're in this talk. What you're like, what is the next phase? The next thing for you is probably something we mentioned prior is to upgrade your game with art, with UI, with surprises. So it doesn't feel like it's something that is prompted out of the gate. So you can up level there. But if you are beyond that as well, then if you're an engineer that cares about scalability, then think about tokens. How do you make sure creators, players, and platform with token economy are going to be profitable? And content safety is one of the bigger challenges. How do you make sure with runtime online, maybe potentially generating images and content, you know that the content is actually safe for your audience. There's a lot of open challenges there. Wherever it is you are, and I mean negative time. So thank you very much for being here from me and David. Thank you for participating and working with the NPCs. Hopefully you had a little fun and go to lunch. Thanks.