Comprehend First, Code Later: The AI Skill I Rely On Daily — Priscila Andre de Oliveira, Sentry

Transcript

[SPEAKER_01] Hello, everyone. Today, I'm going to share with you how I use AI at Sentry and the skill I use the most in my day-to-day work. Before we dive into that, let me tell you who I am. My name is Priscila. I'm a Brazilian based in Vienna, Austria. I'm a mom of a two-year-old, very energetic toddler. I'm a maintainer of Verdaccio, an open-source NPM registry. I'm a co-organizer of Vienna.js, a very traditional meet-up in Vienna, and we talk all about JavaScript. And I'm a senior software engineer at Sentry. Yesterday, someone told me that I don't look like a software engineer, but guess what I am. My title, my official title is senior software engineer, but I have given myself a little promotion, and I am now an agent manager. No salary raise, but at least my reports, they don't complain. Yes, this was me at work a few weeks ago. My colleague, Dominic Dorfmeister, found it funny to see me managing a couple of agents and took this picture. Yeah, luckily, I have three monitors, so that works pretty well. This is my new reality. This is how I feel actually orchestrating a bunch of agents. Yeah, it's weird, but it's fun. And the industry is changing, right? That's why you all are here. And I'm also adapting. Since December 2025, I haven't coded anymore. I'm only prompting. Yes. And even this presentation was created by a skill. I have—yeah, I didn't do anything. So, as you can see, these are some of my recent contributions to Sentry. And I created a few PRs together with my favorite teammate, Claude. And it's not just bug fixes. It's also features, refactors, cross-repository contributions. So it's real. It's working. This is Sentry. Maybe you don't know Sentry, but we are very well known for error and performance monitoring. But we have grown into a full observability platform. We have error monitoring. We have metrics. We have profiling. We also have agentic tools to monitor your agentic platforms. Yes, the code base is very complex. It was founded in 2010. It has 15 plus years of code. We have around 400 employees around the globe. We have 100K organizations depending on this code base working every day. And as an employee, I also depend on this code base working because I get my salary from it, right? So I don't want to just ship bad code. So it's a serious business. And we vibe code as well at Sentry. Recently, we had a hackathon where we could have a few days to just get ourselves familiar with AI and try out new things. And a lot of good projects came out of this hackathon. We have, for example, Abacos. This was created to track the usage of AI internally at Sentry. We have Worden. This is a code review agent. You can have it in your PRs. We have Junior. Junior is a bot we have in our Slack. Because usually people, they see like, oh, I don't like this UI. Something changed. Can you go fix it? Or why was this changed? They like to go and share something, maybe some bug they found in Slack. And then we can just trigger Junior. And Junior can analyze that thread and create a fix for the bug. And people are having a lot of fun with Junior. And there is also this AI SDK testing repository. This was created before this hackathon. But this is basically a repository where we create tests for our AI integrations. And this was really weird because I started contributing to this repository. And my team told me I shouldn't code at all. I should only prompt until I would get a nice result. So it was a different experience. But yeah, it's working. We are using all of these tools internally every day. And at Sentry, we are going all in AI. But quality still matters. Last year, during three months, we used this time as a quality quarter. We used this time just to improve our code bases. Remove all the any types from TypeScript, for example. Or all the to-dos. Usually, I don't know if you guys have this thing, but we had a lot of to-dos—do something else at certain time. And we used this time to simplify code and remove unused feature flags and really have our code base in a good shape. And this is very important, right? This is called technical debt. And as I told you before, the Sentry code base is very complex. And it's a moving target. We have about 100 PRs merged every day. We have four offices. Sentry is open source. So you can contribute to Sentry. We have also contributors. And we are all the time deprecating components, adding new components, adding new lint rules. I don't know, you name it. All the time something happens at Sentry. I've been there over six years now. And I can go on vacation. I come back and maybe my PR is full of conflicts and I have to solve those conflicts. And I really need to understand. All the time I need to align and understand something. It's a daily practice. And this is not new, right? There are studies behind it. 70% of a developer's time is spent reading and navigating code and so forth. This hasn't changed. But now we have a very smart tool which can help us understand faster. And that's how I'm using AI. I'm using AI a lot to understand, to explore and understand. And maybe you may think you just tell AI to go explore the code base and have it do something. But maybe AI understood incorrectly, understood wrongly. And you need to also understand because maybe you need to steer the AI to go on the correct path. And so this is how I'm using AI. This hasn't changed. But now we have a very smart tool which can help us understand faster. And that's how I'm using AI. So I'm using AI a lot to understand, to explore and understand. And maybe you may think you just tell AI to go explore the code base and have it do something. But maybe AI understood the wrong thing, understood wrongly. And you need to also understand because maybe you need to steer the AI to go on the correct path. And yes. So this is how I'm using AI. It made me faster but not the way you think. Maybe because before, let's say an incident happened. And I would have to track that down. I would have to open API, open GitHub, sorry, go git blame and then try to understand where the regression happened. And now I can just prompt a simple phrase and I have it in a few seconds. Or before maybe a product decision. Why did this change? And then I would, I don't know, ask this question in Slack. Maybe my colleague is in another country, another time zone and I would need to wait for that answer the next day. And now I can just ask AI and I have it. So it's been really useful and this made me really productive. But the understanding part of it, yes. And my prompts, they kept repeating. So I had this idea to let Claude analyze my cache and see. So it analyzed 116 sessions and it classified everything in six categories. Comprehension, modification, process, review, generation and order. And guess what? This impressed even me. Oops. So 67% of my AI usage was comprehension. And only 2% code generation. So this, I was very surprised. So because my prompts kept repeating, I created a skill for me. This is a skill, it's locally in my computer. I could share it with someone if I wish. But I use this for me, here. And it's called Catch Me Up. That structures those comprehension questions into six exploration modes. Architecture, convention, feature trace, syntax, testing and history. So, a skill is just a very detailed prompt, right? To very clear goals. I can actually do this here. You can see how it is. But it's just an MD file with human language. And I am a very visual person. I work at Centaur a lot on the front end part of it. And I like to see things to understand. So, this skill brings me the organogram, the structure, a table for me to understand. I think it helps a lot. And I can now give you a short demo. Just a minute. By the way, this is my presentation running. Oh, it's here actually, here. I already run this skill because maybe I would have some issues. But do you remember that project I told you that I should only prompt and not code anymore? So, in the beginning, I was not familiar with that project. It was a new repository for me. And I used my skill for that. I said, I am a new, can you guys see this well? Yes. So, I said, I am a new contributor. Catch me up on how this repository works and clarify whether it simulates a century envelope and intercepts it during tests. I am using Claude as you can see, Opus. And it gave me here a summary. And I like this flow, how it works. And here, also, the answer to my question, does it simulate envelopes? No. It intercepts real ones. They spawn collector. Yes. I'm not going to read it. But this information, all of this, it's very useful. If I didn't have AI, I would have to do this myself, right? I mean, I don't like to just ship something I don't understand. If it's a vibe-coded project, that's fine. But this is serious business. It's my work. So, yes. This skill is helping me a lot with that. And also to review PRs because maybe I'm reviewing a PR of a colleague. I have a lot of context, but not enough to approve that PR. And I wanted to have that context. So, I use this skill to give me that. Okay. So, back to my vibe-coded presentation. So, Jack Nations wrote a blog post called Vibe-coding our way to disaster, drawing on Rich Hickey's simple made easy philosophy. He proposes three phases, research, planning, and implementation. I think you guys also heard about it outside. Even Claude Code has this planning mode, right? So, yes, I agree with all of that. But I think it's missing this step. You need to understand the research your agent did, right? You need to understand that. And to steer, as I said, maybe it's going the wrong direction. Or maybe you need to explore something else. And you need to have that to understand that. Then, after that, you can say, okay, plan that for me. And let's do the implementation. Let's go ahead. So, Armin Ronasha, he's the creator of Flask and a former Centaur. Now, he's working on his own startup. Today, he's going to give a talk here, by the way. He's around. So, he wrote in his blog post, when more and more people tell me they no longer know what code is in their own code base, I feel something is very wrong here. And, yes, I agree. So, what I hope you can take away from this presentation is that the biggest unlock from AI in a large code base isn't generation. It's comprehension. I tracked my own usage and I was surprised. 67% of my prompts are comprehension and only 2% generation. Maybe you track your own AI usage as well and you can improve it, right? So, AI is the teammate who never gets tired of your questions. So, there are no dumb questions. It's the cheapest senior engineer out there. And yes, I agree. So what I hope you can take away from this presentation is that the biggest unlock from AI in a large code base isn't generation. It's comprehension. I track my own usage and I was surprised. 67% of my prompts are comprehension and only 2% generation. Maybe you track your own AI usage as well and you can improve it, right? So AI is the teammate who never gets tired of your questions. There are no dumb questions. It's the cheapest senior engineer out there. So just go for it. Yes. And align your mental model before you prompt because the code is going to flow naturally. And don't ship swap code into the code base that pays your salary. Ship kino code. Really. This is the term the industry is using. And yes, thank you. [SPEAKER_01] So TriCentry, this was a sponsored talk. [SPEAKER_01] We have a booth downstairs. [SPEAKER_01] If you wanted to stop by to say hello. [SPEAKER_01] If you scan this QR code, you're going to get three months free trial of our business plan. [SPEAKER_01] And I hope you enjoyed the presentation. Afterall, let's see. Afterall, let's see. Afterall, let's see. Afterall, let's see. Afterall, let's see.