Ian Silber - What it's like designing at OpenAI

Transcript

I think some people might assume we're two years ahead, thinking like that, but we're running very closely with where all of these advancements are going. And so that's just a very different way of working. Things are changing underneath your feet all day long. And it's very exciting. It's really fun to be, I don't know, we're going to figure this out as we go. We're going to try it. We're going to turn the crank. We're going to keep iterating. We're going to keep going. Welcome to Dive Club. My name is Rid, and this is where designers never stop learning. Today's episode is with OpenAI's head of product design, Ian Silver. So we're going to go deep into all of the ways that they work, what it's like designing with AI as a material, Ian's thoughts on how the role of designer is evolving, and a lot more. But before we get into all of that, I had to know: how the heck does somebody become the head of product design at OpenAI? I had been at Instagram for eight years. It was amazing. Most everything, the big stuff that we'd worked on, I was able to be involved in. Some of my Kevin and Mikey were starting a new company, and I went and joined them at Artifact. It was this sort of news AI thing. A ton of fun, great team, but I had some really good friends that had also left Instagram that were, hey, we're starting this thing. We're building the game. We don't totally know what we're doing, but we'd love for you to come work with us on it too. I don't know, it was at the point in my career where I was, I should just try some crazy stuff. I'm not even a hardcore gamer, or I have zero background in gaming. In that way, I was unqualified. But the team that they're assembling was both really close friends, which is always fortunate if you ever get that opportunity, but also some of the most talented people that I've had the chance to work with in my career, both from design, but also really great engineers. I think we took a lot of lessons from Instagram, which has some game-type mechanics or behavior. So we leaned a lot into the social side and that sort of thing. And it was a game, but it was a little bit of everything. The TLR of what we built was Minecraft in the browser, essentially. But we were trying to combine Minecraft meets Roblox, where Roblox is anybody can create these games. And there's this full marketplace. People are constantly creating games, and then you can come and play them, and people can actually make money off making these games. And it's this whole sort of industry. But when you really go to make the games, it's pretty comp, it's not the most approachable experience. But we loved how approachable Minecraft was, where you're just one-to-one building things. And so we wanted to combine that together and say, what if you could let anybody come into this world and build, but also let other people participate in that? And we built a lot of different mechanics into the game. But one of the things that I really liked was the idea that anybody can create their own version of it and then let other people play that. It was a ton of fun. Working on a game is very different. One thing that was interesting is it's hard to know. Everything's a good idea in some ways. You're, oh, that would probably be fun. And so the amount of scope you can easily get into is quite wide. And we realized why these AAA studios take eight years to release a game. Unless you're making a small mobile game, it's easy to see how sprawling it can really be. So then what's the story? How did you get to OpenAI? So first of all, this was when ChatGPT, or GPT-4, I think, had come out when we're a year into the startup. Our founder went away and came back, and he's, guys, this is changing everything. We need to think about what this means for everything. It wasn't any specific direction, but he was just, this is the real deal. He studied AI in college and has a background and all that stuff. And he was, yeah, this is a true different point in time. We were using it to make the game. And this was so funny, but way back in the day, this was, I guess we had GitHub Copilot, so you could do some autocomplete stuff, and a lot of copying and pasting between ChatGPT and back into VS Code or whatever. I was doing a lot of front end at the time. It was incredible. But it's funny to think how far the workflows, yeah, it's like Stone Age. And it was only two years ago, Stone Age. I was writing all this front end by hand. And what happened was, the true story is that me and the designer had both been reached out to by the head of product at OpenAI at the time, who we had both worked with in the past. He had recently joined OpenAI. He was leading ChatGPT. And he was, look, this thing's going great. We have a great team, but we need to grow it really fast. And we need people that can help us figure out how to get to this next scale. He didn't actually realize that we were working together when he was trying to hire both of us. And so we talked, and he also found out about the engineering team that we had. And one thing led to another, we all ended up joining as a team. There were eight of us. It was a pretty funny time where we went from talking about some random specific game mechanic on a Friday, and then that next Monday, we're deep in the OpenAI trenches trying to figure out what we're launching next and how this whole place works. So, real quick message, and then we can jump back into it. If you're like me, then you're prototyping a lot in code lately. But the problem is, you have this choice between a tool that isn't hooked up to your actual code base and design system, or a localhost prototype that's super annoying to share. That's why I love what Dessin is doing. In one click, you and everyone on your team can prototype directly in your code base without ever opening an IDE. Dessin extracts your design language and gives you the perfect sandbox to explore without any of the technical hurdles. And when you're ready, there's a nice little share button, top right, and you can send it to anybody on your team. It's a big deal, and you can connect your code base and start prototyping today. Just head to dive.club slash dessin. That's D-E-S-S-N. Never in a million years did I think my design workflow would change so drastically in the last few months. And a big part of it is Paper's new snapshot tool. It's a Chrome extension that lets you copy any component or element from your live website and then paste it directly into Paper as editable layers. So all the time, I'll grab something from prod, have Claude immediately spin it up six different ways on Paper's Canvas, and then when I'm ready to send a concept back to Claude, it's seamless because Paper's Canvas uses real HTML and CSS. That workflow feels a lot like the future of design to me. And you can start doing this today. Just head to dive.club slash paper to try it out. Now onto the episode. Before we get all into the present day stuff, I want to talk a little bit about some of those first few months, even as you're simulating. What were some of the defining characteristics that made the OpenAI trenches unique compared to past experience? From day one, it was such a different place. I remember very distinctly my first all-hands. They have these all-hands, and they show you here's where we're at today and here's what we're seeing as possible with the future, where these models are going. And it really just sinks in, the rate of progress and what is coming. From that moment, I was, wow, this place is different. I think the biggest difference is working in a research-led environment. OpenAI started as a research lab, and that DNA is very much there. It's very different than something like Instagram, which I was very familiar with, or maybe more like a company that started as a consumer product. OpenAI started as a research lab and just trying to figure out what they're going to build. And Chat2BT launched just a low-key research preview. And when you think of it through that lens, it's just a lot where we're at today, and here's what we're seeing as possible with the future, where, where, where these models are going. And it really just sinks in, the rate of progress and what is coming. From that moment, I was like, wow, this place is different. I think the biggest difference is working in a research-led environment. OpenAI started as a research lab, and that DNA is very much there. It's very different than something like Instagram, which I was very familiar with, or maybe more a company that started as a consumer product. OpenAI started as a research lab and just trying to figure out what they're going to build. And Chat2BT launched just a low-key research preview. And when you think of it through that lens, it's just a lot of different, there's just a different way that the entire company approaches it. The company's super mission-driven and really thinking about how we develop really far forward into the future. Can we go a little deeper into what it looks like to thrive as a designer when you're working closely with a research lab? How does that shift the practice of design on a daily, weekly basis? So much of our work is figuring out what the models are good at and then trying to wrap that in a product that people can understand and can use. We like our designers to be really close to the model side and really thinking deeply about, and playing with, and trying it, and seeing where it breaks, seeing where it falls down, understanding the mechanics we can use to tweak the behavior. But a lot of it is really just having a ton of curiosity. I think what it really comes down to is you don't have to be technical to work here. But I think you have to be really curious and really interested to see how might somebody actually use this? A lot of times it just gets handed this new thing. And we have to figure out how we can productize that, how we can get that into a thing that people can now go to their computer or their phone and be like, wow, look at this thing it can do. Yeah, it's really just a lot of thinking about the model as the product. I think one thing we try to spend more time on is honestly, a lot of times, what can we do without pixels? Can we do this with tokens? Can we do this with the model itself? And always trying to push to do more of that, what can happen directly in the conversation and in the flow of how you're using the experience versus trying to think about what new bespoke UI do we need for this? There's obviously a balance. And I think there are sometimes where you really want that and sometimes where you don't. It's just a new tool or a new material that I think you have to think about working with. I think for most people listening, they probably don't have as much of a grid for what it looks like to design outside of the pixels. So how deep can we go on that? Maybe is there an example of some of the design decisions or explorations that you all are doing internally that are happening more at the model level? I'll give you one example that we're thinking about right now, which we're still just honestly experimenting with. If you think about how you introduce ChatG to a new user, there's this very traditional onboarding flow that lots of products do. There's this tour and account creation, and we're trying to ask you some questions to learn about you. That's been effective, and people are used to that. But you could think about that totally differently, right? You could say, well, actually, we have this super intelligent model that could probably do a much better job trying to understand what this person's goals are, what they're trying to learn, or what questions they may have, what functionality we might want to tell them about that will help them with whatever tasks they're trying to carry out. We're really stripping back a lot of what you might traditionally do and trying to say, well, actually, let's think about how we should give this context to the model, that this person is brand new and they might need some handholding, or they might need to learn about a specific feature or honestly understand what this thing even is because it's so different than what maybe you've traditionally used before to get information or to learn or whatever. And so how can we let the model do the work versus trying to write a bunch of static explanation of what this thing is. And so there you start to get into thinking about the system prompt and the model behavior and that sort of thing, which is just a totally fun new way of working. And with that, you can quickly prototype, like, here's what it would typically output if we did nothing. Well, what if we gave it a different context, or what if we tried telling it this or that or whatever, and how does that actually change behavior? Does that make it more friendly or easier to use or help people understand better what you can do with this thing? And so I think that's one example where designers working on this are hopefully spending a lot less time in Figma or whatever tool you use to draw pixels and more time really thinking about how you interact with this thing and the fact that the model really is the core product. Is there a set of guiding principles that help you think about when to reach for interface-level solutions and when to outsource it to the model? Because I would imagine there's probably a lot of potential for all kinds of different bespoke interfaces. I know you did the learning experiences now. How do you maintain the simplicity of chat as an interaction paradigm, while also reaching for the full potential of what this experience can be? That's a really good question. We don't have principles. We probably should. I think it's more, I guess at this point, a little bit intuition. You're totally right that just text is definitely not the end-all. And we're seeing that, right? So we have all sorts of things that you can do with chat now, right? You ask it for something, and if you're asking it to write something, instead of it just, yeah, sure, and then it gives you a bunch of text, you know, do you want me to change anything? And you ask and you get a new term. What we did there was we looked at that and we just thought critically. We saw that one of our number one use cases for ChatGPT, and this is a really fun design-led exercise, it's a fun one to talk about. We looked at the data of how people use ChatGPT. A huge percentage of people use ChatGPT to write, to help them write. And then within that, there are lots of different use cases for how you do that. There's a bunch of things that are just funky with it. So for example, there's the classic, you see somebody copy the entire thing and they paste that, and it's an email to somebody, and at the bottom there's, let me know if you want this shorter or something. That's one thing, but also just the interaction loop, right? And if you want to just change one word, it was tedious because you would then have to, okay, change the second paragraph to say this or that or whatever. And so we wanted to lean into more direct manipulation. And so with that, now when you ask it to write, and we're still rolling this out, so for some writing cases you'll see it, others you won't, but we want to get this out to all writing use cases. But now you get this container where it puts what you were writing into the container. And you can still use chat to just say, oh, actually make this longer, make it shorter, and that'll work the traditional way. But you can now select a certain part of text, you can delete it, you can ask for a specific change, and it's just targeting that one thing. And so it's just a way more ergonomic way of working. And so that's an example of where it's a mix of model behavior and working with the model to say, when should it show this? When should it not? And then how do you manipulate stuff directly in this writing block? And then the next thing we think about too, how do we make sure we're building a system so that anytime we add one of these things that feels a little bit different than pure text, how can we make that a system so that we're building these different building blocks so that eventually the model, I think, will be able to compose and pull these together in different bespoke ways specific for what that user's task is at hand. What do you think are some of the traits of the best systems thinkers at OpenAI? It's easy to think about the one use case that you're working on right now. If you think about how people use ChatGPT, it's very fluid, right? You might be a mix of model behavior and working with the model to say, when should it show this? When should it not? And then how do you manipulate stuff directly in this writing block? And then the next thing we think about too, how do we make sure we're building a system so that anytime we add one of these things that feels a little bit different than the pure text, how can we make that a system so that we're building these different building blocks so that eventually the model, I think, will be able to compose and pull these together in different bespoke ways specific for what that user's task is at hand. What do you think are some of the traits of the best systems thinkers at OpenAI? It's easy to think about the one use case that you're working on right now. If you think about how people use ChatGPT, it's very fluid, right? You might be working on one thing and then flowing into something else, or one moment you're asking a question about what to pack for a trip coming up, and then next you're helping it write an email to your boss, and then the next thing, you're doing some research. And so people move between these contexts in very different ways. And so I think the best systems thinkers are thinking not just about their feature, but how does this feature extend the system? One thing we're really trying to think a lot about is what are the underlying primitives that these products need in order to serve the user's needs. And so if you build this once, it's going to make every other thing that we're trying to do better. I think you can see some of these examples that are starting to crystallize around, for example, skills as this new primitive that I think these products are starting to embrace. And the more we can find these, the deepest abstraction that encapsulates what it is you're trying to do, then we can use that for other features or other tasks that the model might want to do. And eventually we want to have a system that not only humans can understand, but a model can reason about and know when and how to use this. I want to talk about what gets shipped and how, because you're in many ways sitting on top of this cauldron of research-led innovation, and all these new capabilities are popping up. It's a very generalizable product that is being used by literally everyone in the world. And so you can go in almost any direction. And so you have all of this experimentation that's happening. Talk to me a little bit about what it looks like to effectively steward an idea as a designer at OpenAI. How do you go from raw experiment to actually getting something like that direct manipulation, or maybe the new learning experiences, out the door? So much of it starts with the prototype or design. It's not just a designer, of course. Anybody has these great ideas: an engineer, a designer, a PM, a researcher. But from a designer's perspective, as everybody knows, it's become much easier to build a working version of something. The best versions of these are a designer will have this idea, and now with Codex or whatever tool you want to use, you can build real versions of this that aren't just clickable prototypes, but are actually live model responses and are working with the model. So I think that opens the door. And so, for example, these math ones, that was a designer who was just thinking about, if you ask for something and it gives you back this LaTeX, the math expressions and all that, it seems pretty archaic. That is not the way that you should be able to learn. You should be able to interact with these things. And so that was a designer who just observed that, put together a version, but then actually tried it with a bunch of things where they said, "Hey, you get the model and you tell the model, if you would typically respond with this, try to respond with this now." I think it was just so clearly valuable when we shared that around that the team rallied around it and eventually helped harden it, helped expand use cases, and ship it. So I think there's a lot of pockets like that. There's a lot of bottoms-up, just in general. That's another thing that I think is different at OpenAI, just tons of bottoms-up stuff. And I think this is probably maybe the way some of the industry is going, because anybody can build an idea now. You can, of course, just fire up Codex or whatever tool and ship it, basically. That's something we think a lot about. We want to empower everybody to have these ideas. And then it becomes, how do you edit and make sure that it all fits together as a system and we're shipping the right things? Looking at when you joined, you mentioned you were copying and pasting from ChatGPT, and now everything just feels like shooting lightning bolts out of your fingertips in many ways. How has that changed the way that teams collaborate, the way that ideas get presented? How have you seen the practice of design shift as a result of this new technology? So when I joined, two and a half years ago, some designers were using Origami. Some designers were, I think, very technical and were really working with the API. For example, they would be in the Playground trying different stuff. I think it wasn't as accessible to designers who were more familiar with traditional design tools. I think you fast-forward to today, obviously, you now have, and this really happened in the last few months, which is crazy to think. Yeah. First you had Cursor, and I think that was cool, but I think even that was a little bit inaccessible to some parts of the team. And now with Codex, and just I think the future of where these products are going, you have these tools that can really go and do real work for you. And it's more about getting your idea out and into it and expressing it properly. There's a whole effort going around to figure out what tooling do we need to make this super effective. So, for example, how do you hook it up to a design system so that it's not just spitting out random UI, but something that feels like it would fit within our system. And so, how do you make the connection between prototyping, production work, and Figma, and all of this? But I think we're seeing more and more of our designers embracing this. It's been really cool to see all of a sudden, instead of a Figma prototype or a static thing, or even a recording of a video, it's very interactive. I can go and just tap around and I can play with it. And actually, in many cases, I can start asking questions, or I can see what the actual model behavior would be. And it's also not just about design, which I think is one thing that's cool. For example, we have an internal agent that we use, which is like a data scientist. Any designer working on something can ask questions. Well, how do people actually use this? Give me the data on how this is used, how this is used, what are the use cases, whatever your question might be. Now you can be way more informed with what decisions you're making. And when you're working at scale, that's really valuable. I love that because it speaks to this lightening of reliance that we have on a bunch of other people to be able to explore and put something out into the world. Everything you're describing is like, man, the amount of times I've been stuck in Metalab or something trying to remember how to do a SQL query. It's like, I just want this answer. It's going to be wild to think, once these things continue to develop and everybody has these, I think everyone will be able to do so much more. And the other thing that is interesting to think about is just understanding when to turn to what toolkit. I think we're still evolving as an industry, right? In some cases, you should definitely be coding a live prototype or using a tool to code to build a live prototype. I think in other cases, a paper sketch is also valuable and the right part of the process. And I think that the discourse has been interesting. I think it's easy to swing one way or the other. But I think sometimes you want a whiteboard, sometimes you want a paper sketch, sometimes you want a wireframe, sometimes you want Figma, and sometimes you want these live prototypes. And I think that the best designers will understand and intuit when to go to which one, because sometimes you want to go super wide. You want to try 1,000 ideas or 100 ideas. And I don't think we have great tools yet that help us do that part, the wide exploration. I think that's one area I'm interested in seeing. I think it will probably evolve quite a bit because you don't In some cases, you should definitely be coding a live prototype or using a tool to code to build a live prototype. I think in other cases, a paper sketch is also valuable and the right part of the process. And I think that the discourse has been interesting. I think it's easy to swing one way or the other. But I think sometimes you want a whiteboard, sometimes you want a paper sketch, sometimes you want a wireframe, sometimes you want Figma, and sometimes you want these live prototypes. And I think that the best designers will understand and intuit when to go to which one, because sometimes you want to go super wide, you want to try 1,000 ideas or 100 ideas. And I don't think we have great tools yet that help us do that part, the wide exploration. I think that's one area I'm interested in seeing, I think will probably evolve quite a bit, because you don't want to just take one prototype, and it's pretty easy to get obsessed with that and just, okay, I want to refine this, and your idea might be way off. I think there's this depth and breadth thing that we need to figure out and not swing too far one way. And I think as a designer, you just need to be really flexible, understanding when to go where, especially as a designer who doesn't have a rich front-end background, and all of a sudden you can make this thing and it feels amazing. And you're like, this is the greatest thing I've ever made in my life because it's fully functional, but it might be completely wrong. A hundred percent. And I started as a hybrid designer engineer. Honestly, dating myself, I think most designers code. We were writing the front-end code at the time. I was really into the 37 Signals crew at the time and that kind of stuff, where you're just really close to building this live experience. I think a lot of people were much closer to code and were using that as a material. You're a lot of designer-engineer hybrids. Of course, you had specialists like iconographers and visual designers and that sort of thing. But the traditional software design, I think, was really playing both roles. And then as the industry grew, I think people started to specialize, and you had this new product design, which was definitely more on the pure design side. And then we're going to have specialist engineers that are going to go and build that. I think that's not changing. It's swinging back, where I think both can overlap way more. And so there's just a big question around what skill set, and where, when do you work, where and how do you work, is shifting quite a bit. Let's pull on that a little bit then, because as somebody in a leadership position, you're having to think a little bit about where is this going? How do we want to work in the future? If you do extrapolate, what does this look like? So when you think about the skills that become more valuable, you mentioned curiosity. Is there anything else that's top of mind for you that you're looking for in designers today? Yeah. In some ways, I think nothing has changed as far as what a great designer is. Our team, I think, does a lot of this, which is just thinking really deeply about what problem are we trying to solve, who are we building this for, and let's try a bunch of different ideas. And how do you converge on an idea? And then of course craft and taste and understanding of interaction patterns and all of that, none of that has changed. And I think it is just as important. I think it's just all this understanding of working with something that is changing very quickly, that is going to evolve. It's not set in stone. It's changing every day, that can kind of shape-shift and do many different things. And then you have these new tools. And so you can express your ideas very differently. Going back to my time at Instagram, which was part of Facebook, I think one thing that really drew me to that company was their deep investment in design tools. At the time when I joined, there was Quartz Composer. They had all this custom stuff they'd built on top of it that Mike Mattis and a few other people had led the charge on. They built Origami. It was Brandon Walkin. And that was so cool to see, that there was this new tool that, when a designer went from Sketch to then using Origami, you could totally see their ideas come across much better. They had to think through not just how it looks, but how does it work, and how does it feel, and how do you move between these states? And I think at the time, as we were shifting from web to mobile, that was really important because so much of it was about how you interact with this thing. And so that tool, I think, did a really good job for that time, that point in time. What I saw is the people that embraced it, I think, were able to uplevel their work and their craft and their impact as a designer. I see that same shift happening now with these new tools that are emerging. I think Codex or Cursor or anything can really unlock a new way of expressing yourself for a designer. Because it's not even just about the visuals. It's also about how it feels. But in your case, it's also like the content itself is designed, which is really interesting. I've never even worked in a place where that is as much of the design as the corner radius that it sits in. Yeah. And it's so interesting because a lot of us have probably worked in places where user-generated content is what people see. So Instagram is kind of a shell, right? When you're designing, you're designing the shell, and then you don't know what content is going to go in there. You don't have control over that. But this is something in between, because we do have some control, actually, over what goes in, but not really. You don't know exactly the way the user is going to use this or approach it or ask, but you can play a little bit with how the model should behave and how it should respond. And so it's somewhere in between this you have no control and you have some control. And so I don't know, it's just totally different. One thing that Dive Club has made abundantly clear to me over the last year is that the practice of design is changing, and the whole process of getting feedback just doesn't quite cut it in today's world. That's why I'm excited to announce that In Flight is officially in open beta. It's the feedback tool that I've always wanted, and it's built for a world that moves at the speed of AI. So I can share my prototypes, give context and video walkthroughs, and In Flight makes it easy to get the exact feedback that I need to move forward, whether it's voting on directions or maybe even getting the green light to ship a new idea. And all of this is available in a single link that I can drop into Slack or maybe even share with power users to test out a new prototype. I use In Flight every day, and it's totally transformed the way that I share work. So I'm excited for you to try the product. And if you ever want to jam about it, just email me at rid@inflight.co. So you mentioned to me that the first two years were kind of holding on for dear life. You're seeing absurd scale, a level of scale that most people don't get to experience. Now that you're kind of on the other end of this, what are some of the more intentional shifts that you're trying to make as a design leader? I'm still holding on for dear life, yeah. But I'm trying to not just let it all come at us as a design team and try to figure out how we can be more intentional about some of the things that we want to work on. I think there is the classic stuff. For example, we didn't really ever have a design systems team because we're moving so fast, and we're establishing that now. I think what's cool about that is we're really trying to think of it, though, from first principles. How do you prototype ideas, and how does it work with the model? So we have a whole system called the Dynamic User Interface Library, which allows us to design things that the model can then interpret. And so that's a new way of thinking about this. And so I think there's that whole thing. There's figuring out what are the systems and tools that you need as a designer so that you can just hit the ground running and do great work in this new way of working. And then of course, there's what is our process? how we can be more intentional about some of the things that we want to work on. I think there is the classic stuff. For example, we didn't really ever have a design systems team because we're moving so fast, and we're establishing that now. I think what's cool about that is we're really trying to think of it, though, from first principles: how you prototype ideas, and how does it work with the model. So we have a whole system called the dynamic user interface library, which allows us to design things that the model can then interpret. And so that's a new way of thinking about this. And so I think there's that whole thing. There's figuring out what are the systems and tools that you need as a designer so that you can just hit the ground running and do great work in this new way of working. And then, of course, there's what is our process? Honestly, it changes day to day. Sometimes we're off in Figma land, and sometimes we're off prototyping things, and we're firing off things over Slack and giving feedback and moving really quickly. But I think we're trying to figure out a slightly more intentional way while keeping ourselves honest that you got to just be willing to roll up your sleeves and try things really fast. And can we add a little bit of clarity there? Let us be a fly on the wall for a week as a designer at OpenAI. How does it work? We have a lot of traditional rituals that, of course, we're always figuring out. A lot of designers will just have an idea and explore it. And we have a channel, for example, that we call PD whip, which is designers' work in progress. You just throw stuff in there. It's got to be a prototype or video or something. We try to make it something that people can really easily react to. That's just a place we've had for a really long time, an open place to just share stuff and get ideas out and riff. We do do crits. And I think that they are generally valuable to get a bunch of people to riff on ideas. And that's really about building ideas. We're still figuring out, do we do design reviews? And if we do, exactly how? And then day to day, I guess it depends on what you're working on. We do have a traditional structure where you have a PM that you're working with and an engineer. And honestly, a lot of this is building and iterating. I think we try to embrace that process of we're going to have to feel this out. We definitely don't spend our time trying to craft the exact perfect solution until we know that it even works. And so I think trying to get to an early version that we can play with in the product is a really good milestone, and then iterating from there and spending a bunch of time talking about, well, is there another way we can do this? Have we looked at how this fits with this and connecting all the dots? Of course, now, because when we ship stuff, we have to be thoughtful about, is this something we want to ship to all of our users? And will this extend the system nicely? And if not, are there other ways or other things that we can do to build out the system so that it all comes together cohesively, which is something we're trying to do more of, to be totally honest. You've talked about the importance of systems thinking multiple times now. So it's obviously a core tenet of what it looks like to design at OpenAI. We talked about what it looks like when it's good and when it's working. Are there pitfalls that you're trying to avoid where maybe it's like, what's the opposite of good systems thinking look like? I think what we're trying to balance, I would say, is how do we put out things that are experimental and early and going to change? That's the research lab nature. But then how do you do that in a way that, for the things that truly matter, feel cohesive? So I think it's more about just understanding the balance of, okay, this is new. We don't totally know, versus this is something that we feel like we really need to harden and we're going to get right. I guess, what does not good systems thinking look like? Honestly, I think it's just if you're a little bit too blinders-on and just trying to say, well, how do we get this thing out as quickly as possible, not totally understanding that we have a bunch of other things going on that are actually pretty similar. And if we all came together, and that's honestly my job. That's probably why I think about this a lot. My job is to try to connect all this stuff and say, well, what if we did less here? Actually, what if there's a way that we actually did fewer of these things, but we did it this way? Maybe that pulls it all together. As I think about the evolution of ChatGPT, that's something that we're trying to figure out: just how do we tighten up a lot of the new things that we're experimenting with and turn it into something that feels like you understand when to go to which tool or how to expose what you should, what you could do with this thing in a way that is clear, as we have new features or new advancements, that we understand that could maybe fit into this system. All while recognizing that, honestly, we don't know because there could be something tomorrow that comes out that completely changes the way you might want to interact with this thing. So you have to be super flexible. Can we talk about that through the lens of the dynamic interface library, then? What are some of the design challenges or opportunities that you all are wrestling with there? How do you design this system so that it can render everywhere natively? How do you figure out how to make it interactive? How do you make sure that it's truly adding value to the experience and not just doing what was done before, but trying to think of the AI, AGI-pilled version of whatever you might be trying to interact with or work with? And then I think the big thing looking forward is how do these things come together, and how does the model eventually have an understanding of it so that maybe it can start to help us compose these without having to have a designer handcraft all of these. We're not quite there yet, but I do think that that will be possible very soon. So when you think about these, you're thinking about these individual components that stack up to a system that a designer can use, an engineer can use, and the model can use. And how does that all come together? It begs some interesting questions about what the future role of a designer even is in that world, where you're loosening all grip on what the interface can be. I think it's an important question. I do think that as all these tools let people make more things, you're going to need an editor. I think a good example, a good analogy, of course, is before the camera, if you wanted a still of somebody, you had to sit down and paint that thing. You had to be able to paint a portrait, and not everybody could do that. If you wanted to, it was time-consuming and expensive. Now, all of a sudden, anybody can take a photo. There are still excellent photographers and people that master the craft and people that have taste. And then there are people that use photography for their personal life, and it doesn't matter. The craft doesn't really matter there. It's about the memories and moments and all of that. There's a similar way you could think about that with the fact that now anybody can write software for anything. I think that just because anybody can do it doesn't necessarily mean it's going to be good or the right thing to build or the right instinct. I think we will, as designers or any function, your job is going to be more and more about helping edit and direct and curate. I don't think the job of a designer is going away anytime soon. I think that, if anything, we'll be able to do more, but I think that the core skills will still be just as important because I think people can recognize good software from bad software. Right. And we all know what that feels like. And it's not just about, does it work, but how it works and how it feels, and is it solving the right problems for me? You've talked about this tension between designing for present capabilities, or maybe the most recent thing that the models have produced, versus the super AGI-pilled part of it. So how do you deal with that tension, and what are some of the more futuristic things that are rattling around your brain right now? One thing we think a lot about is the capability gap. The capability gap about helping edit and direct and curate. I don't think the job of a designer is going away anytime soon. I think that, if anything, we'll be able to do more, but the core skills will still be just as important because people can recognize good software from bad software. Right. And we all know what that feels like. And it's not just about, does it work, but how it works and how it feels, and is it solving the right problems for me? You've talked about this tension between designing for present capabilities or maybe the most recent thing that the models have produced versus the super AGI-pilled part of it. So how do you deal with that tension, and what are some of the more futuristic things that are rattling around your brain right now? One thing we think a lot about is the capability gap. The capability gap is saying that the models actually are now at a point where they can do a lot, right? If you look at codex and what it can do, sometimes it's spending a lot of tokens. It's taking a long time. And then what people use, let's say, ChatGPT for today, right? There's actually now a gap between what the models are actually capable of. And that actually just happened very recently. I think, as designers, we have to think a lot about, well, how do you actually expose that in a way that people understand what they can do with it? And how do you give them the tools so that they can go out and do real meaningful work right in ChatGPT? And so, yeah, I think that's something we think about a ton. How do we shape the product, give you the tools, so you can really take advantage of everything that these models are able to do if they really sit and spend the time to work for you? And we just haven't done that yet, right? There's a limited amount of functionality that you can do inside of ChatGPT. Unlike if you think about codex, codex can go and use your computer and spend a bunch of time and use the different skills and all of that. And again, you always have to remind yourself, so much happened just in three months. What's the next three months going to look like? What's the next? I also try to remind myself of, we're working on this thing three years, or whatever it is, into its existence. When computers first came out and they were three or four years into their existence, or five, whatever, and where they are now. And just each year, how do you have that advancement? And then more and more things were able to be done. It's funny, side note, but there's this really fun podcast, 30 for 30, which is the sports podcast. I love 30 for 30. Yeah, okay, cool. So 30 for 30 is the show, but they also have a podcast. Have you heard the one about Madden? No. Okay. There's this really great episode about Madden football, the game. This is a pillar of my childhood, by the way, so I'm on the edge of my seat. I cannot wait to see how you loop this back in. I just remember the story from that where they'd made this game on NES, I think, or maybe it was even before NES. I don't remember. They wanted Madden to be the spokesperson, and Madden was super into it, but he looked at it and he was like, there's only eight people on the field, or whatever, five people on the field. Football has, how many is it? 11? I don't know. They're like, there are not enough bits to do that. It's technically not possible to put 22 players on the field. We can only put 10 total. And he was like, come back to me when that's possible. Wow. And then a year or two later, the chips advanced, and then all of a sudden they were able to make it. And that was the very basic version of Madden where they finally had the 11 players. And then he was like, cool, I'll endorse it. I want to be part of this. I love it. And then it's a really fun podcast because you hear him in the booth recording his sound bites and stuff like that. But it was a cool story of also thinking about how much technology advances on this regular basis. And we all know this as you're working with these models. They're not perfect. You run into these issues, like, ah, why it falls down in a certain case, or it's really good at this, but it's actually really bad at some other thing. And you just have to have faith that a year from now, or even less, six months, and you extrapolate out where that's going. It's hard to imagine, but I think a lot about that and what it must have been like to design software or build software 20, 30 years ago when there were all these limitations that you're constantly running into and you're working with these details. I mean, we talk about context management and context window and that sort of thing. And I don't know, is that going to feel archaic in five, 10 years? And so even the concept of prompt engineering, I was thinking about the other day, we put so much emphasis on exactly how to format and structure. And you had all those copy-and-paste graphics that would be all over Twitter with the different color-coded. And now I'm like, I don't even think about that anymore. Yeah. It's all changing so quickly. So it's our job to try to think about going back to, well, you do have to deal with the realities of what it is now, because if you go too far ahead, you can't put 11 people on the field. So you can't design the game, but that's trying to bring it back. So you have to find the balance. And that's where really understanding what it's capable of now, and then also being able to push it and say, well, we actually do want to be able to do this. So how can we make sure we're trying to help set some vision for that? Is there anything that you're doing intentionally as a leader to help the design org push on that future or be more generative or more risk-taking in what they're exploring? I think it's a good question and probably something we need to be more clear about, when to do it. I think that's the big question, finding when the right time is to do that. I think in some cases you want to do it; in other cases you don't. And I think what I'd like to do is find more time where you can flip-flop between doing both. I think that's something we're honestly trying to figure out as a team. How much time should I be spending doing that kind of work versus more execution? And I think you have to be able to be very fluid. We've covered a lot of ground. What have we not talked about yet that you think paints a picture or shines an accurate light on what it looks like to design at OpenAI and the culture that you have and just the way that you all operate? The reality is that it is very fast-paced. We are figuring things out quickly. We try to update our own thinking very quickly. And we have to evolve quickly with the technology and as a team. And so I think some people might assume we're two years ahead, thinking like that, but we're running very closely with where all of these advancements are going. And so that's just a very different way of working. Things are changing underneath your feet all day long. And it's very exciting. It's really fun to be like, I don't know, we're going to figure this out as we go. We're going to try it. We're going to turn the crank. We're going to keep iterating. We're going to keep going. I want to push on something that you said earlier. You talked about how, in many ways, being a great designer hasn't changed. You're solving problems. But yet a lot of the things that you are talking about do feel almost more reactive to the technology rather than user behaviors. And I have to imagine that that throws a little bit of a wrench into the design practice that maybe you would have put at Instagram or something like that. Sure. So what I should say, to make it clear, I think that great designers can do both. Great designers can both say, oh, wow, we have this new technology. Okay, cool. How do we package that up? But then I think other designers, or other times our designers, are thinking about, well, actually, it needs to do this. It needs to be really good at this. Or here's how this could work. And it's just a blend now. And you just have to think about one more thing. Honestly, it's like you have to think not just about the problems, the user problems, and all talking about do feel almost more reactive to the technology rather than user behaviors. And I got to imagine that that throws a little bit of a wrench into the design practice that maybe you would have put at Instagram or something like that. Sure. So what I should say, I think, to make it clear, I think that great designers can do both. Great designers can both say, oh, wow, we have this new technology. Okay, cool. How do we package that up? But then I think other designers, or other times, designers are thinking about, well, actually, it needs to do this. It needs to be really good at this. Or here's how this could work. And it's just a blend now. And you just have to think about one more thing. Honestly, it's like you have to think not just about the problems, the user problems, and all that, but you have to think about what the technology can't do that it should be able to do. So you have to be able to do all that. So I think it's just extending one part of the process. And I think the best process for designers is thinking, well, how do we push this forward? How do we make this be more useful if it could just do X, Y, and Z, or here's an ideal way this might work. And then we can work with engineering or research or whatever to get it there. Okay. So for somebody listening, who's inspired by the conversation, they want to join the team, you'll be hiring throughout this year, I'm sure. Yeah. What are some of the main signals that you would be hunting for in a design candidate? And what would you be doing to figure out if they're present and if they're the type of person that would thrive in this environment? I think there's just a few candidate types that we always look for. I'm always very excited about the more up-and-coming people that just have tons of energy. I think that it's funny. I used to say when I first joined two, two and a half years ago, you don't need a background in AI to come work here. You have to be curious about the technology. And that's still generally true. But I think that there's enough now, there's been enough time, where finding people that have started to experiment and play with this stuff and understand what it's good at, what it's bad at, where it needs to go, and how we need to push on the tools and the technology and the products that we're all building. I think there's enough there to play with. I've always respected people that will go deep on some side project or get really passionate about some idea. You need the fundamentals. You need to be great at doing general product design. But I think that other piece, just being truly curious and interested in the idea. And ideally you really spent time playing with it and understand, and hopefully you have lots of ideas for where we can push things. We're at this fortunate place where you're not just reacting to the technology, but you're hopefully helping shape where this is going and helping to expand the capabilities that we can put into the product, into people's hands, so that they can do more with it. And so people that are excited about that kind of way of working. Well, Ian, I appreciate you coming on and sharing this with us today. You all have had a massive impact on state of design and even interacting, just creating these paradigms that we're all building on top of. And so it's been fun to hear a little bit of the behind the scenes and how you all operate today. Yeah, this was super fun. Thanks for having me. when to turn to what toolkit, I think we're still kind of evolving as an industry, right? In some cases, you should definitely be coding a live prototype or using a tool to code to build a live prototype. I think in other cases, a paper sketch is also valuable and right part of the process. And I think that the discourse has been interesting. I think it's easy to swing one way or the other. But I think like sometimes you want a whiteboard, sometimes you want a paper sketch, sometimes you want a wireframe, sometimes you want Figma, and sometimes you want these live prototypes. And I think that the best designers, I think will understand and intuit when to go to which one, because sometimes you want to go super wide, you want to try 1000 ideas or 100 ideas. And I don't think we have great tools yet that help us do that part, the kind of wide exploration. I think that's one area I'm interested in seeing, I think will probably evolve quite a bit because you don't want to like just take one prototype and you can it's pretty easy to kind of like get obsessed with that and just like, how do we like, okay, I want to refine this and your idea might be way off. I think there's like this depth and breadth thing that we need to figure out and not swing too far one way and just be I think as a designer, you just need to be really flexible, understanding when to go where, especially as a designer who doesn't have a rich front end background, and all of a sudden you can make this thing and it feels amazing. And you're like, this is the greatest thing I've ever made in my life because it's fully functional, but it might be completely wrong. A hundred percent. And like, you know, I started as a hybrid designer engineer. I mean, honestly, like dating myself, I think most designers code like we're writing the front and code at the time, you know, I was like really into the 37 signals crew at the time and that kind of stuff where you're just like really close to building this live experience. I think most, a lot of people were like much closer to code and we're kind of using that as a material, you're kind of like a lot of designer engineer hybrids. Of course, you had specialists and like iconographers and visual designers and that sort of thing. But like the traditional kind of like software design, I think was like really kind of like playing both roles. And then as the industry grew, I think people started to specialize and you had this new kind of like product design, which was definitely more on the just pure design. And then we're going to have specialist engineers that are going to go and build that. I think that like, that's not changing. It's kind of swinging back where I think both can overlap way more. And so there's just a big question around what skill set and where, when do you work where and how do you work is just like shifting quite a bit. Let's pull on that a little bit then, because as somebody in a leadership position, you're having to think a little bit about like, where is this going? How do we want to work in the future? If you do extrapolate, what does this kind of look like? So when you think about the skills that become more valuable, you mentioned curiosity. Is there anything else that's top of mind for you that you're looking for in designers today? Yeah. You know, in some ways I think nothing has changed as far as like what a great designer is. Our team, I think does a lot of this is just thinking really deeply about like, what problem are we trying to solve? Who are we building this for? And let's try a bunch of different ideas. And like, how do you kind of like converge on an idea? And then of course craft and taste and, you know, understanding of interaction patterns and all of that, like, none of that has changed. And I think is just as important. I think it's just all this understanding of working with something that is changing very quickly, that is going to evolve. It's not like set in stone. It's like, it's changing every day that can kind of shape shift and do many different things. And then just like, yeah, you have these new tools. And so you can, you can express your ideas very differently. Going back to my time at Instagram, which was part of Facebook. I think one thing that was, what really drew me to that, to that company was their deep investment in design tools. At the time when I joined as Quartz composer, they had like all these like custom kind of stuff they'd built on top of it that like Mike Mattis and, and, and a few other people had kind of like, I think led the charge on they built origami. It was like Brandon Walken. And that was like, so cool to see that there was this like new tool that when a designer went from sketch to then using origami, like you could totally see their ideas come across much better. They had to think through not just how it looks, but how does it work? And how does it feel? And how do you move between these states? And I think at the time, as we were shifting from web to mobile, and I think that was really important because so much of it was about like, how you interact with this thing. And so that tool, I think did a really good job for that time, that point in time. What I saw is the people that embraced it, I think we're able to uplevel their, their work and their craft and their impact as a designer. I see that same shift happening now with like these new tools that are emerging. I think codex or cursor or anything can really unlock like a new way of expressing yourself for a designer. Because it's not even just about the visuals. It's also about how it feels. But in your case, it's also like the content itself is designed, which is really interesting. Like I've never even worked in a place where that is as much of the design as the corner radius that it sits in. Yeah. And it's so interesting because a lot of us have probably worked in places where like user generated content is kind of what people see. So like Instagram is kind of a shell, right? You're kind of like when you're designing, you're designing the shell, and then you don't know what content is going to go in there. You don't have control over that. But this is like something in between, because we do have some control actually over what goes in, but not really. Like, you know, you don't know what the exactly the way the user is going to kind of use this or approach or to ask, but you can, you can play a little bit with how the model should behave and how it should respond. And so there's like, it's somewhere in between this kind of like, you have no control and you have some control and you know, and so I don't know, it's just totally different. One thing that dive club has made abundantly clear to me over the last year is that the practice of design is changing and the whole process of getting feedback just doesn't quite cut it in today's world. That's why I'm excited to announce that in flight is officially in open beta. It's the feedback tool that I've always wanted, and it's built for a world that moves at the speed of AI. So I can share my prototypes, give context and video walkthroughs and in flight makes it easy to get the exact feedback that I need to move forward, whether it's voting on directions or maybe even getting the green light to ship a new idea. And all of this is available in a single link that I can drop into Slack or maybe even share with power users to test out a new prototype. I use in flight every day and it's totally transformed the way that I share work. So I'm excited for you to try the product. And if you ever want to jam about it, just email me at rid at inflight.co. So you mentioned to me that the first two years we're kind of holding on for dear life. You're seeing absurd scale, like a level of scale that most people don't get to experience. Now that you're kind of on the other end of this, like what are some of the more intentional shifts that you're trying to make as a design leader? I'm still holding on for dear life, yeah. But I'm trying to not just let it all kind of come at us as a design team and try to figure out how we can be more intentional about some of the things that we want to work on. I mean, I think there is the classic stuff. Like for example, we didn't really ever have a design systems team because we're moving so fast and we're establishing that now. I think what's cool about that is we're really trying to think of it though, from first principles, how you prototype ideas and how does it work with the model. So we have a whole system called the dynamic user interface library, which allows us to design things that the model can then interpret. And so that's like a new way of thinking about this. And so I think there's that whole thing. There's kind of figuring out like, what are the systems and tools that you need as a designer so that you can just like hit the ground running and and do great work in this new way of working? And then of course, there's what is our process? Honestly, like it changes day to day. Sometimes we're off in Figma land and sometimes we're off like, you know, prototyping things and we're firing off things over Slack and giving feedback and moving really quickly. But I think we're trying to figure out a slightly more intentional while keeping ourselves honest that you got to just be willing to just like roll up your sleeves and try things really fast. And can we add a little bit of clarity there? Like, let us be a fly on the wall for a week as a designer at OpenAI. Like, how does it work? I mean, we have a lot of traditional rituals that of course we're trying to, we're always figuring out. A lot of designers will just have an idea and explore it. And we have a channel, for example, that we call PD whip, which is like designers just work in progress. You just throw stuff in there. It's got to be a prototype or video or something like we try to make it like something that people can like really easily react to. That's just like a place we've had for a really long time, like an open place to just kind of share stuff and just like get ideas out and riff. We do do crits. And I think that they are generally valuable to like just get a bunch of people to kind of like riff on ideas. And that's really about just kind of like building ideas. We're still figuring out, do we do design reviews? And if we do exactly how, and then like day to day, I mean, I guess it depends on what you're working on. We do have a traditional kind of structure where you have like a PM that you're working with and an engineer. And honestly, a lot of this is like building and iterating. I think we try to embrace that process of like, we're going to have to feel this out. We definitely don't spend our time, I think, like trying to craft the exact perfect solution until we know that it even works. And so I think trying to get to an early version that we can play with in the product, I think it's like a really good milestone. And then iterating from there and spending a bunch of time talking about, well, like, is there another way we can do this? Have we looked at how this fits with this and connecting all the dots? Of course, now, like, because when we ship stuff, we have to be thoughtful about, is this something we want to ship to all of our users? And will this extend the system nicely? And if not, are there other ways or other things that we can do to kind of like build out the system so that it all kind of comes together cohesively, which is something we're trying to do more of, to be totally honest. You've talked about the importance of systems thinking multiple times now. So it's obviously like a core tenet of what it looks like to design at OpenAI. We talked about what it looks like when it's good and when it's working. Are there pitfalls that you're trying to avoid where maybe it's like, what's the opposite of good systems thinking look like, you know? I think what we're trying to balance, I would say, is how do we put out things that are experimental and early and going to change? That's like the research lab nature. But then how do you do that in a way that for the things that truly matter feel cohesive? So I think it's more about just understanding the balance of like, okay, this is new. We don't totally know versus like, this is something that we feel like we really need to harden and we're going to get right. I guess what is not good systems thinking look like? You know, honestly, I think it's just if you're a little bit too blinders on and just trying to say, well, how do we get this thing out as quickly as possible, not totally understanding that we have a bunch of other things going on that are actually pretty similar. And if we all came together, and that's honestly my job. That's why I think probably why I think about this a lot is like my job is to kind of try to connect all this stuff and say, well, what if we did less here? Actually, what if, what if there's a way that like, we actually did fewer of these things, but we did it this way, maybe that pulls it all together. As I think about the evolution of ChatGPT, that's something that we're trying to figure out is just like, how do we tighten up a lot of the new things that we're kind of experimenting with and turn it into something that feels like you understand when to go to which tool or how to expose what you should, what you could do with this thing in a way that is clear as we have new kind of features or new advancements, like we understand that that could maybe fit into this system. All while recognizing that honestly, we don't know because like there could be something tomorrow that comes out that completely changes the way you might want to interact with this thing. So you have to be super flexible. Can we talk about that through the lens of the dynamic interface library then? What are some of the design challenges or opportunities that you all are wrestling with there? How do you design this system so that it can render everywhere natively? How do you figure out how to make it interactive? How do you make sure that it's truly adding value to the experience and not just like doing what was done before, but trying to think of like the AI kind of AGI-pilled version of whatever you might be trying to kind of interact with or work with? And then I think the big thing looking forward is how do these things come together and like how does the model eventually kind of have an understanding of it so that maybe it can start to help us compose these without having to have a designer handcraft all of these. We're not quite there yet, but I do think that that will be possible very soon. So when you think about these, you're thinking about these like individual components that kind of stack up to a system that a designer can use, an engineer can use and the model can use. And how does that all kind of come together? It begs some interesting questions about what the future role of a designer even is in that world, where you're kind of loosening all grip on what the interface can be. I mean, I think it's an important question. I do think that as all these tools let people make more things, like, you know, you're going to need an editor. I think a good example, like a good analogy, of course, is like before the camera, if you wanted a still of somebody, you had to sit down and paint that thing. You had to be able to paint a portrait and not everybody could do that. If you wanted to, it was, you know, time consuming and expensive. Now, all of a sudden, anybody can take a photo. There are still excellent photographers and people that like master the craft and people that have taste. And then there's, you know, people that use photography for their personal life. And it doesn't matter, like the craft doesn't really matter there. It's about the memories and moments and all of that. There's a similar way you could think about that with the fact that now anybody can write software for anything. I think that just because anybody can do it doesn't necessarily mean it's going to be good or the right thing to build or the right instinct. I think we will, as designers or any function, I think like your job is going to be more and more about kind of helping edit and direct and curate. I don't think like the job of a designer is going away anytime soon. I think that if anything, I think we'll be able to do more, but I think that the core skills will still be just as important because I think people can recognize good software from bad software. Right. And we all know what that feels like. And it's not just about, does it work, but like how it works and how it feels and is it solving the right problems for me? You've talked about this tension between designing for like present capabilities or maybe the most recent thing that the models have produced versus, you know, the super AGI pilled part of it. So like, how do you deal with that tension and what are some of the more futuristic things that are rattling around your brain right now? One thing we think a lot about is the capability gap. The capability gap is saying that the models actually are now at a point where they can do a lot, right? If you look at codex and what it can do, sometimes it's spending a lot of tokens. It's taking a long time. And then what people use, let's say, ChatGPT for today, right? There's actually now a gap between what the models are actually capable of. And that actually just happened very recently. I think as designers, we have to think a lot about, well, how do you actually expose that in a way that people understand what they can do with it? And how do you give them the tools so that they can go out and do real meaningful work right in ChatGPT? And so that's, yeah, I think that's something we think about a ton. How do we shape the product, give you the tools so you can really take advantage of everything that these models are able to do if they really sit and spend the time to work for you? And we just haven't done that yet, right? There's limited amount of functionality that you can do inside of ChatGPT. Unlike if you think about codex, codex can go and use your computer and spend a bunch of time and use the different skills and all of that. And again, you always have to remind yourself, so much happened just in three months, what's the next three months going to look like? What's the next? I also try to remind myself of, you know, we're working on this thing is three years or whatever it is into its existence. When computers first came out and there were three or four years into their existence or five, whatever, you know, and where they are now. And just each year, how do you have that advancement? And then more and more things were able to be done. It's funny, side note, but there's this really fun podcast, 30 for 30, which is like the sports podcast. I love 30 for 30. Yeah, okay, cool. So 30 for 30 is like the the show, but they also have a podcast. Have you heard the one about Madden? No. Okay. There's this really great episode about Madden football, the game. This is a pillar of my childhood, by the way. So I'm edge of my seat. I cannot wait to see how you loop this back in. I just remember the story from that where they'd made this like game on NES, I think, or maybe it was even before NES. I don't remember. They wanted Madden to be the spokesperson and Madden was super into it, but he looked at it and he was like, there's only eight people on the field or whatever, five people on the field. Like football has, how many is it? 11? I don't know. They're like, there are not enough bits to do that. It's technically not possible to put 22 players on the field. We can only put 10 total. And he was like, come back to me when that's possible. Wow. You know, and then like a year or two later, like these, the chips advanced and then all of a sudden they were able to make it. And that was like the very basic version of Madden where they finally had the 11 players. And then he was like, cool, I'll endorse it. Like, I want to be part of this. I love it. And then it's a really fun podcast because you hear him like in the booth recording like his sound bites and stuff like that. But it was a cool story of just like also thinking about how much technology advances, like just on this regular basis. And I mean, we all know this as you're working with these models, they're not perfect. Like you run into these issues, you know, like, ah, like why it falls down in a certain case, or, you know, it's really good at this, but it's actually like really bad at some other thing. And, and you just have to have faith that a year from now or even less six months or, you know, and, and you extrapolate out where that's going. It's like kind of hard to imagine, but I think a lot about like that and, and what it must have been like to design software or build software 20, 30 years ago when there was like all these such like limitations that you're constantly running into and you're working with these details. I mean, we're, you know, we talk about like context management and context window and that sort of thing. And like, I don't know, is that going to like feel archaic and, and five, 10 years, you know? And so even the concept of prompt engineering, I was thinking about the other day, like we put so much emphasis on exactly how to format and structure. And you had all those like copy and paste graphics that would be all over Twitter with the different color coded. And now I'm like, I don't even think about that anymore. Yeah. It's all changing so quickly. So it's our job to kind of try to think about going back to like, well, you do have to be, deal with the realities of what it is now, because if you go too far ahead, you can't put 11 people on the field. So like you can't design the game, but that's trying to bring it back. So you have to kind of find the balance. And that's where like, really understanding what it's capable of now. And then also like being able to push it and say, well, we actually do want to be able to do this. So, so how can we like make sure we're trying to help set some vision for that? Is there anything that you're doing intentionally as a leader to help the design org push on that future or be more generative or more risk taking in what they're exploring? I think it's a good question and probably something we need to be more clear about when to do it. I think that's the big question is finding when the right time is to do that. I think in some cases you want to do it in other cases you don't. And I think what I'd like to do is find more time where you can kind of flip flop between doing both. I think that's something we're kind of honestly trying to figure out as a team. Like how much time should I spend be spending doing that kind of work versus more execution. And I think you have to be able to be very fluid. We've covered a lot of ground. What have we not talked about yet that you think paints a picture or shines an accurate light on what it looks like to design at OpenAI and the culture that you have and just the way that you all operate? The reality is that it is very fast paced. We are figuring things out quickly. We kind of try to update our own thinking very quickly. And we have to evolve quickly with the technology and as a team. And so I think some people might assume we're like two years ahead thinking like that, but we're running very closely with where all of these advancements are going. And so that's just like a very different way of working. Things are changing underneath your feet all day long. And it's very exciting. It's really fun to be like, I don't know, we're going to figure this out as we go. We're going to try it. We're going to turn the crank. We're going to keep iterating. We're going to keep going. I kind of want to push on something that you said earlier. You talked about how in many ways, being a great designer hasn't changed. You're solving problems. But yet a lot of the things that you are talking about do feel almost more reactive to the technology rather than user behaviors. And I got to imagine that that throws a little bit of a wrench into the design practice that maybe you would have put at Instagram or something like that. Sure. So what I should say, I think to make it clear, I think that great designers can do both. Great designers can both say, oh, wow, we have this new technology. Okay, cool. How do we package that up? But then I think other designers or other times our designers are thinking about, well, actually, it needs to do this. It needs to be really good at this. Or here's how this could work. And it's just a blend now. And you just have to think about one more thing. Honestly, it's like you have to think not just about the problems, the user problems and all that, but you have to think about what the technology can't do that it should be able to do. So you have to be able to do all that. So I think it's basically just extending one part of the process. And I think the best process for designers is thinking like, well, how do we push this forward? How do we make this be more useful if it could just do X, Y, and Z, or here's kind of an ideal way this might work. And then we can like work with engineering or research or whatever to like get it there. Okay. So for somebody listening, who's inspired by the conversation, they want to join the team, you'll be hiring throughout this year, I'm sure. Yeah. What are some of the main signals that you would be hunting for in a design candidate? And what would you be doing to figure out if they're present and if they're the type of person that would thrive in this environment? I mean, I think there's just a few candidate types that we always look for. I'm always very excited about the kind of more up and coming people that just have tons of energy. I think that it's funny. I used to say like when I first joined two, two and a half years ago, you don't need a background in AI to like come work here. You have to be curious about the technology. And that's still generally true. But I think that there's enough now, there's been enough time where finding people that have started to experiment and play with this stuff and understand what it's good at, what it's bad at, where it needs to go and how we need to like push on these, on the tools and the technology and the products that we're all building. I think there's like enough there to play with. I've always respected people that will go deep on some side project or get really passionate about some idea. You need the fundamentals. You need to be great at just kind of like doing, you know, general product design. But I think that that other piece, just like being truly curious and interested in the idea. And ideally you really spent time playing with it and understand like, and hopefully you have lots of ideas for, for where we can push things. We're at this fortunate place where you're not just reacting to the technology, but you're, you're hopefully helping shape where this is going and like helping to expand the capabilities that we can put into the product into people's hands so that they can do more with it. And so people that are excited about, about that kind of way of working. Well, Ian, I appreciate you coming on and sharing this with us today. You all have had a massive impact on state of design and even interacting, like just creating these paradigms that we're all building on top of. And so it's been fun to hear a little bit of the behind the scenes and how you all operate today. Yeah, this was super fun. Thanks for having me.