AI Engineer

Comprehend First, Code Later: The AI Skill I Rely On Daily — Priscila Andre de Oliveira, Sentry

914 summary words 4 min summary Watch video

Start with the signal

4 min read

Summary

30-second take

A Sentry senior engineer reveals she hasn't written code since December 2024—only prompts—and tracked 116 AI sessions to discover 67% of her AI usage is comprehension, not generation (2%). Her thesis: in large, moving codebases (Sentry has 15 years of code, 100 PRs/day, 400 employees), AI's biggest unlock is understanding context fast—tracking down regressions, understanding why decisions were made, reviewing unfamiliar PRs—not churning out code. She built a custom "Catch Me Up" skill with six exploration modes (architecture, convention, feature trace, syntax, testing, history) to structure repeated comprehension queries. The talk pushes back on pure vibe-coding hype: you must understand the research your agent does before planning/implementing, or you'll ship garbage into production codebases that pay your salary.

Key takes

  • 67% comprehension, 2% generation: Tracking actual AI usage over 116 sessions revealed the speaker spends the vast majority of time using AI to understand code (architecture, conventions, feature trace, history), not to generate it—contradicting the common assumption that AI is mainly a code-writing tool.
  • Large codebases = moving targets: Sentry's 15-year codebase sees ~100 PRs merged daily across four offices and open-source contributors; going on vacation means returning to merge conflicts and needing constant alignment—AI speeds up the understanding loop (git blame, incident tracking, "why did this change?") from hours/days to seconds.
  • Custom comprehension skill: The speaker formalized repeated prompts into a local "Catch Me Up" skill with six modes, producing visual outputs (organograms, tables) to scaffold understanding before touching code—treating AI as a structured onboarding/context tool rather than a black-box code generator.
  • Vibe-coding risks without comprehension: Referencing Jack Nations and Rich Hickey, she argues research → planning → implementation misses a critical step: you must understand the research your agent did to steer it correctly and avoid shipping "swap code" (low-quality, context-ignorant code) into production.
  • Quality quarter precedent: Sentry spent three months removing any types, TODO comments, unused feature flags, and simplifying code—technical debt hygiene matters when AI agents multiply contributions, because agents inherit and amplify existing code quality (or lack thereof).

Useful details

  • Sentry context: Founded 2010, 400 employees globally, 100K organizations depend on it; full observability platform (error monitoring, metrics, profiling, agentic tools).
  • Internal AI projects from hackathon: Abacos (tracks internal AI usage), Worden (code review agent), Junior (Slack bot that triages bugs/UI complaints and creates fixes), AI SDK testing repo (prompt-only contributions).
  • Stat: 70% of developer time is reading/navigating code (cited study, pre-AI baseline).
  • Demo example: Asked AI "I am a new contributor, catch me up on how this repository works and clarify whether it simulates a Sentry envelope" → AI provided summary, flow diagram, answer ("No, it intercepts real ones, they spawn collector").
  • Prompt-only workflow: Since December 2024, zero manual coding—PRs include bug fixes, features, refactors, cross-repo contributions via Claude Opus.
  • Quote (Armin Ronacher, Flask creator, former Sentry): "When more and more people tell me they no longer know what code is in their own code base, I feel something is very wrong here."
  • Terminology: "Kino code" = high-quality code (industry term speaker uses vs. "swap code").

Caveats / counterpoints

  • Selection bias: The speaker works at Sentry, which has invested heavily in AI tooling and ran a dedicated hackathon—this environment may not generalize to orgs without mature AI infrastructure or cultures resistant to prompt-only workflows.
  • No failure examples: The talk shows successful PRs and demos but omits cases where AI misunderstood requirements, produced incorrect context, or required multiple steering iterations—cost/time tradeoffs are unaddressed.
  • "Only prompting since December 2024": No discussion of skill atrophy, debugging edge cases, or situations where hands-on coding is faster (e.g., trivial fixes, hot paths requiring performance tuning).
  • Custom skill replication: The "Catch Me Up" skill is local and not shown in full (only an MD file glimpse)—unclear how much prompt engineering is needed to replicate similar structured comprehension for other codebases.
  • Quality quarter as prerequisite: She mentions Sentry spent three months cleaning up debt before AI adoption scaled—unclear if the comprehension-first approach works in messier, unmaintained codebases or if it requires upfront hygiene.

Ken relevance

  • Agent systems design: The 67/2 split (comprehension/generation) suggests Ken's agent systems should prioritize context retrieval and explanation layers over raw code output—agents that scaffold understanding (organograms, trace flows, history) may deliver more value than blind code generators.
  • AI ops workflow: The "Catch Me Up" skill model—formalized, repeatable prompts with structured outputs—maps to Ken's thinking on agent memory and task decomposition; consider building similar comprehension skills into agent orchestration for onboarding, PR review, incident triage.
  • Content/GTM angle: "Comprehend first, code later" is a contrarian, data-backed frame against vibe-coding hype—useful positioning for thought leadership, blog posts, or case studies targeting eng leaders worried about AI code quality.
  • Investing lens: Companies building LLM-powered dev tools that focus on comprehension/context (code search, architecture visualization, review assistants) over autocomplete may have defensible moats in enterprise codebases; Sentry's internal tools (Abacos, Worden, Junior) show demand.
  • Personal workflow: Ken should track his own AI usage by category (comprehension vs. generation) to identify high-leverage prompt patterns and formalize them into reusable skills—mirrors Priscila's self-audit method.

Watch verdict

Skim. The 67% comprehension stat and "Catch Me Up" skill framework are immediately useful, but the demo is lightweight and the talk lacks depth on failure modes or replication details. Read the transcript for the key insight; watching adds minimal visual value beyond seeing the skill output UI for ~30 seconds.

Search this video Resume transcript at 0%
Full transcript 2077 words · 10 min read
0:00

[SPEAKER_01] Hello, everyone. Today, I'm going to share with you how I use AI at Sentry and the skill I use the most in my day-to-day work.

0:14

SPEAKER_01

Before we dive into that, let me tell you who I am. My name is Priscila. I'm a Brazilian based in Vienna, Austria. I'm a mom of a two-year-old, very energetic toddler. I'm a maintainer of Verdaccio, an open-source NPM registry. I'm a co-organizer of Vienna.js, a very traditional meet-up in Vienna, and we talk all about JavaScript. And I'm a senior software engineer at Sentry. Yesterday, someone told me that I don't look like a software engineer, but guess what I am. My title, my official title is senior software engineer, but I have given myself a little promotion, and I am now an agent manager.

1:03

SPEAKER_01

No salary raise, but at least my reports, they don't complain. Yes, this was me at work a few weeks ago. My colleague, Dominic Dorfmeister, found it funny to see me managing a couple of agents and took this picture. Yeah, luckily, I have three monitors, so that works pretty well. This is my new reality. This is how I feel actually orchestrating a bunch of agents. Yeah, it's weird, but it's fun. And the industry is changing, right? That's why you all are here. And I'm also adapting. Since December 2025, I haven't coded anymore. I'm only prompting. Yes. And even this presentation was created by a skill. I have—yeah, I didn't do anything.

2:15

SPEAKER_01

So, as you can see, these are some of my recent contributions to Sentry.

2:24

SPEAKER_01

And I created a few PRs together with my favorite teammate, Claude. And it's not just bug fixes. It's also features, refactors, cross-repository contributions. So it's real. It's working. This is Sentry.

2:51

SPEAKER_01

Maybe you don't know Sentry, but we are very well known for error and performance monitoring. But we have grown into a full observability platform. We have error monitoring. We have metrics. We have profiling. We also have agentic tools to monitor your agentic platforms. Yes, the code base is very complex. It was founded in 2010. It has 15 plus years of code. We have around 400 employees around the globe. We have 100K organizations depending on this code base working every day. And as an employee, I also depend on this code base working because I get my salary from it, right? So I don't want to just ship bad code. So it's a serious business.

3:46

SPEAKER_01

And we vibe code as well at Sentry. Recently, we had a hackathon where we could have a few days to just get ourselves familiar with AI and try out new things. And a lot of good projects came out of this hackathon. We have, for example, Abacos. This was created to track the usage of AI internally at Sentry. We have Worden. This is a code review agent. You can have it in your PRs. We have Junior. Junior is a bot we have in our Slack. Because usually people, they see like, oh, I don't like this UI. Something changed. Can you go fix it? Or why was this changed? They like to go and share something, maybe some bug they found in Slack. And then we can just trigger Junior.

4:47

SPEAKER_01

And Junior can analyze that thread and create a fix for the bug. And people are having a lot of fun with Junior. And there is also this AI SDK testing repository. This was created before this hackathon. But this is basically a repository where we create tests for our AI integrations. And this was really weird because I started contributing to this repository. And my team told me I shouldn't code at all. I should only prompt until I would get a nice result. So it was a different experience. But yeah, it's working. We are using all of these tools internally every day. And at Sentry, we are going all in AI. But quality still matters.

5:37

SPEAKER_01

Last year, during three months, we used this time as a quality quarter. We used this time just to improve our code bases. Remove all the any types from TypeScript, for example. Or all the to-dos. Usually, I don't know if you guys have this thing, but we had a lot of to-dos—do something else at certain time. And we used this time to simplify code and remove unused feature flags and really have our code base in a good shape. And this is very important, right? This is called technical debt. And as I told you before, the Sentry code base is very complex. And it's a moving target. We have about 100 PRs merged every day. We have four offices. Sentry is open source.

6:41

SPEAKER_01

So you can contribute to Sentry. We have also contributors. And we are all the time deprecating components, adding new components, adding new lint rules. I don't know, you name it. All the time something happens at Sentry. I've been there over six years now. And I can go on vacation. I come back and maybe my PR is full of conflicts and I have to solve those conflicts. And I really need to understand. All the time I need to align and understand something. It's a daily practice. And this is not new, right? There are studies behind it. 70% of a developer's time is spent reading and navigating code and so forth. This hasn't changed.

7:30

SPEAKER_01

But now we have a very smart tool which can help us understand faster.

7:34

SPEAKER_01

And that's how I'm using AI. I'm using AI a lot to understand, to explore and understand. And maybe you may think you just tell AI to go explore the code base and have it do something. But maybe AI understood incorrectly, understood wrongly. And you need to also understand because maybe you need to steer the AI to go on the correct path. And so this is how I'm using AI. This hasn't changed. But now we have a very smart tool which can help us understand faster. And that's how I'm using AI. So I'm using AI a lot to understand, to explore and understand. And maybe you may think you just tell AI to go explore the code base and have it do something.

8:36

SPEAKER_01

But maybe AI understood the wrong thing, understood wrongly. And you need to also understand because maybe you need to steer the AI to go on the correct path. And yes. So this is how I'm using AI. It made me faster but not the way you think. Maybe because before, let's say an incident happened. And I would have to track that down. I would have to open API, open GitHub, sorry, go git blame and then try to understand where the regression happened. And now I can just prompt a simple phrase and I have it in a few seconds. Or before maybe a product decision. Why did this change? And then I would, I don't know, ask this question in Slack.

9:32

SPEAKER_01

Maybe my colleague is in another country, another time zone and I would need to wait for that answer the next day. And now I can just ask AI and I have it. So it's been really useful and this made me really productive. But the understanding part of it, yes. And my prompts, they kept repeating. So I had this idea to let Claude analyze my cache and see. So it analyzed 116 sessions and it classified everything in six categories.

10:07

SPEAKER_01

Comprehension, modification, process, review, generation and order.

10:12

SPEAKER_01

And guess what? This impressed even me. Oops.

10:31

SPEAKER_01

So 67% of my AI usage was comprehension. And only 2% code generation. So this, I was very surprised. So because my prompts kept repeating, I created a skill for me. This is a skill, it's locally in my computer. I could share it with someone if I wish. But I use this for me, here. And it's called Catch Me Up. That structures those comprehension questions into six exploration modes. Architecture, convention, feature trace, syntax, testing and history. So, a skill is just a very detailed prompt, right? To very clear goals. I can actually do this here. You can see how it is. But it's just an MD file with human language. And I am a very visual person.

11:38

SPEAKER_01

I work at Centaur a lot on the front end part of it. And I like to see things to understand.

11:48

SPEAKER_01

So, this skill brings me the organogram, the structure, a table for me to understand.

11:59

SPEAKER_01

I think it helps a lot.

12:04

SPEAKER_01

And I can now give you a short demo. Just a minute. By the way, this is my presentation running. Oh, it's here actually, here. I already run this skill because maybe I would have some issues. But do you remember that project I told you that I should only prompt and not code anymore? So, in the beginning, I was not familiar with that project. It was a new repository for me. And I used my skill for that. I said, I am a new, can you guys see this well? Yes. So, I said, I am a new contributor. Catch me up on how this repository works and clarify whether it simulates a century envelope and intercepts it during tests.

13:03

SPEAKER_01

I am using Claude as you can see, Opus. And it gave me here a summary. And I like this flow, how it works. And here, also, the answer to my question, does it simulate envelopes? No. It intercepts real ones. They spawn collector. Yes. I'm not going to read it. But this information, all of this, it's very useful. If I didn't have AI, I would have to do this myself, right? I mean, I don't like to just ship something I don't understand. If it's a vibe-coded project, that's fine. But this is serious business. It's my work. So, yes. This skill is helping me a lot with that. And also to review PRs because maybe I'm reviewing a PR of a colleague.

13:54

SPEAKER_01

I have a lot of context, but not enough to approve that PR. And I wanted to have that context.

14:04

SPEAKER_01

So, I use this skill to give me that. Okay. So, back to my vibe-coded presentation. So, Jack Nations wrote a blog post called Vibe-coding our way to disaster, drawing on Rich Hickey's simple made easy philosophy. He proposes three phases, research, planning, and implementation. I think you guys also heard about it outside. Even Claude Code has this planning mode, right? So, yes, I agree with all of that. But I think it's missing this step. You need to understand the research your agent did, right? You need to understand that. And to steer, as I said, maybe it's going the wrong direction. Or maybe you need to explore something else.

14:50

SPEAKER_01

And you need to have that to understand that. Then, after that, you can say, okay, plan that for me.

14:58

SPEAKER_01

And let's do the implementation. Let's go ahead. So, Armin Ronasha, he's the creator of Flask and a former Centaur. Now, he's working on his own startup. Today, he's going to give a talk here, by the way. He's around. So, he wrote in his blog post, when more and more people tell me they no longer know what code is in their own code base, I feel something is very wrong here. And, yes, I agree. So, what I hope you can take away from this presentation is that the biggest unlock from AI in a large code base isn't generation. It's comprehension. I tracked my own usage and I was surprised. 67% of my prompts are comprehension and only 2% generation.

15:59

SPEAKER_01

Maybe you track your own AI usage as well and you can improve it, right? So, AI is the teammate who never gets tired of your questions. So, there are no dumb questions. It's the cheapest senior engineer out there. And yes, I agree. So what I hope you can take away from this presentation is that the biggest unlock from AI in a large code base isn't generation. It's comprehension. I track my own usage and I was surprised. 67% of my prompts are comprehension and only 2% generation. Maybe you track your own AI usage as well and you can improve it, right? So AI is the teammate who never gets tired of your questions. There are no dumb questions.

16:33

SPEAKER_01

It's the cheapest senior engineer out there. So just go for it. Yes.

16:58

SPEAKER_01

And align your mental model before you prompt because the code is going to flow naturally. And don't ship swap code into the code base that pays your salary. Ship kino code. Really. This is the term the industry is using. And yes, thank you. [SPEAKER_01] So TriCentry, this was a sponsored talk. [SPEAKER_01] We have a booth downstairs. [SPEAKER_01] If you wanted to stop by to say hello. [SPEAKER_01] If you scan this QR code, you're going to get three months free trial of our business plan. [SPEAKER_01] And I hope you enjoyed the presentation. Afterall, let's see. Afterall, let's see. Afterall, let's see. Afterall, let's see. Afterall, let's see.

Reading tools

Type to find a passage

Appearance
Ask this transcript

Notebook

Export Markdown

Add a note