AI Engineer

Vibe Engineering Effect Apps — Michael Arnaldi, Effectful

860 summary words 4 min summary Watch video

Start with the signal

4 min read

Summary

Vibe Engineering Effect Apps — Michael Arnaldi, Effectful

Main Topics

  • AI-Driven Development: Using LLMs (Claude, GPT-5.4, Opus) to write production code without manual coding
  • Effect Framework Patterns: Building HTTP APIs, databases, and type-safe systems using the Effect library
  • Repository Architecture for AI: Structuring codebases to optimize AI model performance
  • Context Management: Techniques for managing LLM context windows and preventing model drift
  • Code Quality with AI: Using linters, type systems, and patterns to enforce best practices

Key Points

LLM Capabilities and Limitations

  • No Continuous Learning: Unlike humans, LLMs don't learn from daily interactions. They're trained once, then post-trained, then frozen
  • Context Window Constraints: Models operate on fixed-size context windows (even 1M tokens can confuse models); information gets compressed
  • Generalization Over Memorization: Models work best when they can extract patterns and generalize, not memorize documentation

Repository Strategy: "Clone the Fucking Repo"

  • Core Principle: Instead of relying on LLMs' training data, clone dependency repositories into your project
  • Why This Works:
  • Coding models are optimized to read user code, not node_modules
  • gitignored files are deprioritized by agents like Cursor
  • Direct code access allows models to discover patterns better than MCP servers or documentation
  • Applies Across Languages: Works in TypeScript, Rust, and other languages

Practical Workflow

Phase 1: Pattern Discovery

  • Ask model to explore cloned repositories
  • Generate markdown pattern files (patterns.sql.md, patterns.http-api.md, etc.)
  • These become reference documents for the model

Phase 2: Agent Configuration

  • Create agents.md listing available commands and restrictions
  • Explicitly forbid watch mode/dev servers (prevents hanging)
  • Reference patterns and cloned repositories
  • Set TypeScript diagnostics to "error" for strict enforcement

Phase 3: Spec-Driven Development

  • First: Discuss architecture, generate markdown spec
  • Second: Implement based on spec (using repeated fresh sessions to manage context)
  • Avoid long-running sessions; use RALF loops (fresh sessions per small task)

AI Model Comparison

GPT-5.4 (Preferred)

  • More concise output
  • Better for most coding tasks
  • No usage restrictions

Claude Opus/Sonnet

  • Better for UI work
  • Can make risky shortcuts if allowed
  • Requires strict linter rules to prevent patterns like as unknown as X
  • Requires uppercase prompts to get attention

LLM Optimization Through Constraints

Less is More: Reduce tool access; single-tool approaches (execute) outperform multi-tool systems

Lint Rules as Guardrails:

  • Ban type assertions (as), force schema validation instead
  • Prohibit any and unknown
  • Enforce branded types for IDs (prevent string ID confusion)
  • Force validation at API edge, not in handlers
  • These rules emerged from watching what models naturally produce

Testing Patterns:

  • Use it.layer, it.effect, it.describe for Effect tests
  • Avoid custom wrappers
  • Self-cleanup with transactions for database tests

Project Setup Example

Built a full working TODO API in one session:

  • TypeScript + Bun + Vitest
  • Effect framework with SQL/SQLite
  • HTTP API with OpenAPI documentation
  • Type-safe schemas
  • Proper layer-based testing

Notable Quotes

> "I have not brought any line of code by hand for a while... I'm most likely just going to start to insult the model. And it's fun because it cannot really answer you back."

> "If you're doing a 3 million context, that really means we have to architect around a dumb process... machine that had knowledge of six months ago at best."

> "The model is good when it can operate a large-scale codebase using patterns and doesn't fail at scale. The zero-to-one problem is not really a problem for the first ten hours."

> "Models treat them like a human brain, but they are very different. We learn continuously... With LLMs, this does not happen."

> "This is why I wanted to call this workshop just clone the fucking repo."

> "Many times less is more... the dumbest thing ever ends up working better."

Takeaways

For AI-Assisted Development

  • Don't rely on LLM training data — clone your dependencies into the project
  • Manage context ruthlessly — use fresh sessions for discrete tasks
  • Create reusable patterns — document best practices in markdown for models to reference
  • Automate pattern generation — tools should create patterns once per day, evaluating against evals

For Code Quality

  • Use linters as feedback loops — every pattern the model repeats incorrectly → new lint rule
  • Type system as enforcement — use brands, schemas, validation at edges
  • Spec-first development — write specs in markdown before implementation

For Framework Adoption

  • Generate patterns by exploring source code, not docs
  • Update agents.md continuously as you discover issues
  • Consider workflows/clustering for AI apps with long-running operations (AI calls take minutes, not milliseconds)

For Fine-Tuning Strategy

  • Open-weight models lag frontier models by 3-6 months
  • Evals are difficult (code quality is subjective); combine automated checks + human review
  • Pattern standardization is model-dependent (GPT vs Claude prompting differs)

Future Considerations

  • Workflows become critical when integrating AI (long response times = high failure rates)
  • Effect.Cluster enables distributed task guarantees across crashes
  • Temporal relevance: AI integrations make workflow solutions essential for companies at any scale
Search this video Resume transcript at 0%
Full transcript 9931 words · 66 min read
0:14

SPEAKER_00

So welcome everybody. Just setting up the context for this workshop, I had a lot of ideas to potentially prepare but at the end I thought we are vibe engineering for this to be authentic it has to be from scratch. So I actually prepared absolutely nothing. That means we can take any path that we want and let's hope this is real enough. First of all I'd like to know from the crowd do you already know a fact? Do you know zero? What's your level of familiarity with AI tooling and some kind of questions like that. Lucky enough we're not too many so I hope this can be as interactive as possible. Maybe let's just start. I know Chris. Hello, hello. Hi.

0:31

SPEAKER_00

Familiarity with AI and with effect? Some? Okay good. Running v4 in production. Against advice by the way. Hi Colin. I've never actually used effect but I know. Good. I'm not going to start with the platform. I'm not going to start with the platform.

0:46

SPEAKER_00

Yeah I've been familiar with the agency work here. I think I've been, I haven't been writing code by hand since the beginning of this year. Doing everything in data agent. And the reason I'm using effect is it's encouraging so much safety. So my agents cannot become small cannons. And the reason I want to use effect is because it was possible that we had one API client. I transferred the API client from regular focus client to effect. Then we use the CLI on the public. And now I'm more interested with how you make the effect more discovered by the agents. Okay. I saw the idea about having the record long and software. Not pretty cool. Good. Good. Well, what about you? Yeah. I'm quite familiar with the code code and the links up. But I've not heard much about the effect yet. So it sounds good. Sounds good.

0:52

SPEAKER_00

[SPEAKER_02] Good. So pretty heterogeneous crowd. All interested in some sort of how to use agents effectively with effect pun intended. You pointed out at a very good thing which is cloning, giving the agent access to the repository. And in reality, this session should just be called just clone the fucking repo and get and be done with it. And really, I've also have not been coding by hand since about late this summer. So it's been quite a while.

1:00

SPEAKER_00

[SPEAKER_02] I started programming when I was 12 years old. So it's quite an odd feeling to get to the point where you're no longer writing code by hand. And most of what I do is library level coding. So it's pretty low level, usually fairly complex type machinery stuff that used to require a very good understanding of the language of how the user interacts with the language. And then it's really and so on and so on and so forth, not diminishing in any way, up level development, just that the way you treat a language, if you have to be the library versus the way you treat the language, if you are building an app on top of it, is usually very different.

1:05

SPEAKER_00

[SPEAKER_01] Now sometimes in Upland you have the same requirements as library land, [SPEAKER_03] especially when you need to generalize, abstract over some patterns, make them repeatable, remove the verbosity from the repetitions and so on and so forth.

1:17

SPEAKER_00

[SPEAKER_02] So there's some crossing there, but I definitely thought that AI would be more useful in Upland and I didn't see much usage at library level land and I was dead wrong. Because I'm not writing code by hand. I have not brought any line of code by hand for a while. And I've done that in TypeScript, I've done that in Rust. And the funny element is, given I mostly write libraries, I usually interact with code bases that have zero documentation, that have zero best practices available online. And so I couldn't really use the usual, let's just add an MCP server to get access to the documentation, or hoping that the models have been trained on the documentation enough to be directly useful. And the reality is, with LLMs, people treat them like a human brain, but they are very different. We learn continuously. This is a learning experience. Once we get out of this room, hopefully you're going to know a little bit more from the starting point when you come in. And your brain will keep going and will internalize more and more patterns over time. Then you go to sleep, your brain cleans up a little bit of the mess of irrelevant information that you got during the day. And there's this whole process of transforming experience, the world that we experience every day, into long-term memory. With LLMs, this does not happen.

1:24

SPEAKER_00

With LLMs, you get a pre-training phase where LLMs are trained on all the world of existing knowledge. Usually they get trained on the whole internet. Then they get specialized in some tasks. And then there's the whole post-training phase where models are fine-tuned to act on specific things. For example, coding agents are generic models that have been reinforced, that have had passes of reinforcement learning to operate on code bases. The whole post-training phase of a large language model dedicated to coding is letting the model rip through code bases and having evaluations that tells the training phase how is the model performing. [SPEAKER_01] Is it doing good?

1:45

SPEAKER_00

Is it doing bad? Does the code compile after this change? Does the code fail to compile after this change? And so on and so forth. But once that is done, it's done. There's no more knowledge that comes into the model every day. So if you interact with a model today and you tell something to the model and you say, hey, I want you to do this in a very specific way, tomorrow it's not going to remember. So how do you make it remember? That is the big question. And models, you have to think of them like you're chatting with them, but the reality is you are basically appending messages onto a fixed size array, which is called the context window. And context window is limited. Now there are models with a 1 million tokens context window. And that's not necessarily a good idea because the context window of the model is what is pushed to the neural network and the neural network is going to try to predict what's coming next. So if you push more information, there's a very good chance you're going to confuse the model, which is why a 1 million context window is not necessarily helpful, especially if you're doing multiple things in the same context. So if you're doing a 3 million context, that really means we have to architect around a dump process. We have to architect around some machine that had knowledge of six months ago at best, that is not going to remember everything because even if you have 1 trillion parameters in the model, or even if you have 10 trillion parameters in the model, that's not enough to store all the human knowledge. So you're always going to get compressed knowledge. And in the best case scenario, you have some ability of generalization in the model so that the model can say, hey, I know A, B, maybe I can do C because it's similar to A and B.

1:52

SPEAKER_00

So if you're doing a 3 million context, that really means we have to architect around a dump process. We have to architect around some machine that had knowledge of six months ago at best, that is not going to remember everything because even if you have 1 trillion parameters in the model, or even if you have 10 trillion parameters in the model, that's not enough to store all the human knowledge. So you're always going to get compressed knowledge. And in the best case scenario, you have some ability of generalization in the model so that the model can say, hey, I know A, B, maybe I can do C because it's similar to A and B.

2:28

SPEAKER_00

And you have some form of this emergent behavior and capability of reasoning on new problems. But models have become very good. I've said it myself, I'm not writing code by hand since at a minimum six or eight months. So that means even if the machine is done, it's already at the point where we can leverage it to do good.

3:11

SPEAKER_02

[SPEAKER_00] But how do we do it?

3:27

SPEAKER_00

Now, if the assumption is the model has outdated knowledge, we need a way for the model to get new knowledge.

3:56

SPEAKER_02

[SPEAKER_00] And we said that those models that we use for coding have been reinforced, have gone through reinforcement learning and have been trained in the model to be able to understand your own code base, make changes in your own code base, and replicate patterns that exist in your own code base.

4:27

SPEAKER_03

[SPEAKER_00] So that means that they have been trained on reading human documentation.

4:41

SPEAKER_01

[SPEAKER_00] They haven't been trained on using an MCP server that they've never seen.

4:47

SPEAKER_03

[SPEAKER_00] They've been trained primarily to consume and produce code.

4:52

SPEAKER_02

[SPEAKER_00] So eight months ago, I was thinking, what if I just give the model access to code? [SPEAKER_00] That means if I want to use effect, I'm going to add the effect repository in my directory, just masquerading the effect code base as my own code base. [SPEAKER_00] And maybe I can trick the model into thinking that it's just one big code base. [SPEAKER_00] And that it would explore it and would progressively use it to build up the required knowledge and to clone the patterns. [SPEAKER_00] And there's various ways of doing that. [SPEAKER_00] One could argue the model already has access to library code by having it in NPM in node modules.

5:25

SPEAKER_02

[SPEAKER_00] But coding agents have been trained to focus on your own code, not on the code that is on node modules. [SPEAKER_00] So if you have it in node modules, the model is de-optimized.

5:41

SPEAKER_02

[SPEAKER_00] It's not going to look at it with the same frequency as it looks at your own code. [SPEAKER_00] If you have it in a gitignored directory, the models have been trained not to look at files that are gitignored. [SPEAKER_00] For example, cursor does not index stuff that is gitignored. [SPEAKER_00] So there are all of those random restrictions that we figure out while developing. [SPEAKER_00] And the only way I found the models to be good, regardless of the language, regardless of what you use, is if you just clone the repo, which is the point of this workshop. [SPEAKER_00] So this is a completely empty project.

6:21

SPEAKER_02

[SPEAKER_00] I have some ideas of where we could take this. [SPEAKER_00] My idea would be to set up a Bun repository, use Vitest for testing, use, build up some kind of HTTP server, ideally providing an OpenAPI documentation for consumption, build a type safe client to interact with the backend. [SPEAKER_00] Hopefully, if we have enough time, I'm not sure, tap into the world of workflows and clustering for persistent operations in the backend. [SPEAKER_00] And really, I have nothing set up. [SPEAKER_00] So how do I usually start? [SPEAKER_00] Well, I would like you to set...

6:52

SPEAKER_00

I start nice with the model. But as soon as it derails, you're going to see I'm going to start to insult the model.

7:07

SPEAKER_02

[SPEAKER_00] And it's fun because it cannot really answer you back. [SPEAKER_00] If you don't like the answer, you can just shut it down. [SPEAKER_00] It's not that a human that gets offended. [SPEAKER_00] Maybe. [SPEAKER_00] I would like to set up a project using Bun. [SPEAKER_00] The project should also include setup of Vitest and TypeScript.

7:47

SPEAKER_01

[SPEAKER_00] Check.

7:48

SPEAKER_00

Script. I'm using GPT 5.4. When I started this journey, I was using Sonnet 4. There's many differences between Sonnet 4 and GPT 5.4. Mainly, Sonnet 4 felt a kid with a knife running through the house. That's an example that comes from Geoffrey Huntley, the author of the Ralph Loops. But even as a kid running through the house with a knife, it was still enough to do coding. And now we have models like Opus 4.5, GPT 5.4 that are much, much better. But a very interesting element to think about is OpenWeights models are lagging behind by three to six months compared to frontier models. Sonnet 4.5.

8:23

SPEAKER_00

Which means now we already have models in the open that are smarter compared to Sonnet 4. Which I already used in library level development. How long would it take for those OpenWeight models to become good enough to be used in our daily operations? I don't know. It's just one thought that lately I have more and more. Especially because Anthropic is putting arbitrary restrictions on how we use their models. So I don't really want to use Anthropic models. OpenAI is good for now. Who knows what they're going to do in a year or two. And I open source, of course. Okay, I don't have a git repo created. Create an empty git repo.

9:38

SPEAKER_00

And by the way, if you have questions, if you want to interrupt me, this is supposed to be interactive. I'm here to entertain you for another hour and a half. Initialize git repo. Okay, this is done. It's amazing that using GPT 5.4 with open code would create by default a cloud.md. I think this is from Bun. Yeah, let's trash this. It absolutely has nothing to do with this project. Perfect marketing strategy. Plus they said it wasn't a real fool and two days after they announced Mitas as the new model. Okay, create a source and test directory.

10:45

SPEAKER_00

Let's see what created. Okay, TypeScript. Bundler mode no limit.

11:07

SPEAKER_00

That's fine. Strict. Skip lib check.

11:27

SPEAKER_00

That's fine. And implicit override. [SPEAKER_01] That's good. [SPEAKER_01] Yes. [SPEAKER_01] Also, actually move the files in the profile directory. Moving the entry file. Good. Seems smart enough. Runs a basic smoke test. Okay. Okay, create a source and test directory.

12:34

SPEAKER_00

Let's see what created. Okay, type spun. Bundler mode no limit. That's fine. Strict. Skip lib check. That's fine. [SPEAKER_01] And implicit override. [SPEAKER_01] That's good. [SPEAKER_01] Yes. [SPEAKER_01] Also, actually move the files

13:46

SPEAKER_00

in the profile directory. Moving the entry file. Good. Seems smart enough. Runs a basic smoke test. Okay. So that's a good starting point. We verified with bund run test, bund run type check. Good. We want to add effect beta. We're going to use effect v4.

14:30

SPEAKER_00

It's not yet released for production usage, except he uses in production already.

14:42

SPEAKER_00

So if I have any problem, I'm going to ask you.

14:46

SPEAKER_00

It's fine.

14:52

SPEAKER_00

It's not small, right? Yes. It's effect small. Small because it used to be small and evolved to become bigger. Still very, very thin bundle size. Okay, 1% 14k. There's plenty of context left. [SPEAKER_01] Want to add effect beta.

15:44

SPEAKER_00

[SPEAKER_01] And we want to use effect vtest to write vtest. [SPEAKER_01] I will. I will. That's next. That's next. And speaking of that, I want to try to use the TSGO version of it. I never used this. One point is I haven't used it. So I don't know how to use it. I'm not sure if this is going to work or not. I'm not sure if this is going to work or not. [SPEAKER_01] I'm not sure how it works with the TSGO. [SPEAKER_01] But we don't want to install the TSGO or the FAQs with preview extension. [SPEAKER_01] Oh, the actual base compiler. [SPEAKER_01] If you can use the FAQs repository, you can use the TSGO version of the TSGO.

17:00

SPEAKER_00

[SPEAKER_01] So it's going to be a little bit more. [SPEAKER_01] Yes. [SPEAKER_01] I don't know if Matija... [SPEAKER_01] Allowed... [SPEAKER_01] It's going to be a type script, but it doesn't really use it anymore.

17:19

SPEAKER_00

[SPEAKER_01] Yeah, the point is it does not use it. [SPEAKER_01] And we could just do an alias install. [SPEAKER_01] So install TypeScript as something else. [SPEAKER_01] But I'm not sure if he did it.

17:42

SPEAKER_00

[SPEAKER_01] Maybe let's follow the normal practice. [SPEAKER_01] Let's install TypeScript Go instead of TypeScript. [SPEAKER_01] Would it be able to do this?

18:06

SPEAKER_00

[SPEAKER_01] Who knows?

18:11

SPEAKER_00

[SPEAKER_01] We will find out. [SPEAKER_01] Is this the package? [SPEAKER_01] No, I don't think this is the package. [SPEAKER_01] I think they just stole my crypto wallet. [SPEAKER_01] Except I do not have one. [SPEAKER_01] Check from here.

18:24

SPEAKER_01

Oh! [SPEAKER_00] The NPM package named TypeScript Go is only a place holder security package. [SPEAKER_00] So I use the real preview compiler that provides the TSGO binary. [SPEAKER_00] Well, that was probably a good idea.

18:39

SPEAKER_00

Script TSGO no emit.

18:52

SPEAKER_00

Let's see. One. Exact. [SPEAKER_01] Let's get all found. [SPEAKER_01] Okay.

19:04

SPEAKER_00

[SPEAKER_01] Okay. [SPEAKER_01] Type check. [SPEAKER_01] One. [SPEAKER_01] One. [SPEAKER_01] Type check. [SPEAKER_01] Okay. [SPEAKER_01] Okay. Set up. We ask code to use. Yes. Go. [SPEAKER_01] V.T. [SPEAKER_03] As go.

19:52

SPEAKER_00

[SPEAKER_03] LSP. We work.

19:59

SPEAKER_01

[SPEAKER_03] Maybe. [SPEAKER_03] Yes.

20:14

SPEAKER_01

[SPEAKER_00] Yes.

20:15

SPEAKER_00

That I need. To be able to do. There we go. Native preview.

20:24

SPEAKER_00

That's it. [SPEAKER_01] Okay.

20:41

SPEAKER_00

[SPEAKER_01] I did install that. [SPEAKER_01] Do I need to reload the window?

21:08

SPEAKER_00

[SPEAKER_01] Most likely.

21:14

SPEAKER_00

[SPEAKER_01] Just go.

21:14

SPEAKER_01

Okay. Maybe it worked. [SPEAKER_00] Then let's go here. [SPEAKER_00] And. Yep. [SPEAKER_00] I should be able to do that. [SPEAKER_00] But also there is a nice.

21:48

SPEAKER_01

[SPEAKER_00] And it will not be loaded. [SPEAKER_00] If files are specified. On command line. Nor config to skip this error. What? I'm going to feed it to the agent. In a minute. Okay. Okay. [SPEAKER_00] Okay.

22:53

SPEAKER_01

Okay.

23:10

SPEAKER_01

It's ban that gives issues. Probably, maybe not. Yes, it is ban.

23:20

SPEAKER_01

Then let me stop this. Select the tsconfig to configure this one, this other is package.json, installing dev dependencies, select all. What is this?

23:35

SPEAKER_01

That's VS code, that's fine.

23:42

SPEAKER_01

This needs a lot of work.

23:44

SPEAKER_00

[SPEAKER_01] Do we have the effect better installed? [SPEAKER_01] Oh gosh. [SPEAKER_01] Where is this coming from? [SPEAKER_01] Who knows? [SPEAKER_01] Okay, okay, okay. [SPEAKER_01] One install. [SPEAKER_01] Okay, that's installed.

24:09

SPEAKER_01

Let's see if it catches anything. I think import add from effect, add some x8, 100.

24:15

SPEAKER_01

No, that's a dangling effect. That should be.

24:20

SPEAKER_01

Okay. Do we need to do the post-install before we leverage service to open? I think I've done that. You mean the prepare one? Yeah, that one.

24:27

SPEAKER_00

[SPEAKER_01] Yeah, I did. [SPEAKER_01] Maybe I need to reload. [SPEAKER_01] Reload after that. [SPEAKER_01] Yes, it was easy.

24:33

The Windows solution just restarted. [SPEAKER_01] Okay, so we have it. Probably, maybe not. Yes, it is banned. Then let me stop this. Select the tsconfig to configure this one, this other is package.json, installing dev dependencies, select all.

24:41

SPEAKER_00

What is this?

24:42

SPEAKER_03

That's VS code, that's fine.

25:05

SPEAKER_03

This needs a lot of work.

25:05

SPEAKER_00

Do we have the effect better installed? Where is this coming from? Okay, okay, okay. One install. Okay, that's installed.

25:14

SPEAKER_00

Let's see if it catches anything. I think import add from effect, add some x8, 100. No, that's a dangling effect.

25:21

SPEAKER_01

That should be. Okay.

25:26

SPEAKER_01

Do we need to do the post-install before we leverage service to open? I think I've done that.

25:36

SPEAKER_01

You mean the prepare one?

25:40

SPEAKER_01

Yeah, that one. Yeah, I did.

25:43

SPEAKER_00

Maybe I need to reload. Reload after that.

25:54

SPEAKER_01

Yes, it was easy.

25:55

SPEAKER_00

The Windows solution just restarted.

26:00

SPEAKER_00

Okay, so we have it.

26:18

SPEAKER_00

And now we want to... We have some diagnostic severity to suggestion, warning, and so on and so forth.

26:23

SPEAKER_01

For AI, we would like to turn everything into an error so that the LLM cannot pass, cannot accept code that has any remote resemblance to an error. So this is a project where we will use AI a lot. But we want all diagnostics available to be set to error.

26:43

SPEAKER_01

Okay. [SPEAKER_00] [SPEAKER_00] [SPEAKER_03] [SPEAKER_00] What is the model doing? Wow. Did it update the tsconfig? It did not. Oh, I'm updating the tsconfig, okay. Do you want to use the effect solutions? No. No. And that's another interesting point. The effect solutions, there is a website called effect.solutions. It's a really nice website. Keith Langton did this. And it's a quick start to use effect in an AI project. And it does install the language service and strict policy defaults and so on and so forth. But then it uses a CLI to give the model access to the effect repo. And the model needs to know how to use the CLI. So it's a dog chasing its tail.

27:11

SPEAKER_01

The CLI is more for the access basically what I was in this documentation. So what's on the website? That is exposed to the right. In fact, it was another repo. Yes. Yes. But there are some markdown files. But it doesn't work as well. And if you actually read at some point, it says you should actually just clone the repository. Okay, this looks exactly what I had in mind. So we have all the diagnostics set to error. Which is good. It's exactly what we want. Reload window. Okay. I also want to format on save to true. Just because it's annoying otherwise. Okay, very good point. Commit current. Commit current. Now I want to add effects more as a subtree.

27:11

SPEAKER_01

Okay, it's committed. Good. Now create dot repos folder. And add as a git subtree without history squashed in repos effect. At least it did. Okay, here. Why is it trying to.

27:16

Okay. [SPEAKER_01] We have it. Let's just check git log.

27:24

Yep. I did audit. [SPEAKER_03] Okay.

27:57

[SPEAKER_00] And now we are at the point where we can start to do our research.

28:28

For example.

28:39

We said we want to create an HTTP API. I would clean up this.

28:51

Open a new session. To avoid context pollution.

28:56

You have access to the effect repository.

29:04

At repos. Actually let's do something else before we want.

29:11

We want to set up an agents.md.

29:18

Set up an agents.md. [SPEAKER_03] Listing the commands. Available.

29:35

Like. One.

29:45

[SPEAKER_00] Run. Type.

29:54

Check. And. Specify. That you have access to the effect repository. At repos/effect. And you should use that to extract best practices.

30:17

Look at how things work.

30:23

[SPEAKER_01] Etc. Now the agents.md. Now we're going to get an initial prototype. As you work in the project. You're going to evolve that.

31:17

SPEAKER_01

You're going to add more commands to it. You're going to add rules. When you spot that some bad patterns are created. In code. One thing we have not set up yet. Is a linter. Linter is going to be an essential. Piece of the back pressure loop. That helps the model drive in the right direction. If you want a fully working setup. I have a repository of mine. That I use for fun. Which is called accountability. In this repository. You can find. A lot of things. [SPEAKER_02] But for example. I have an ESLint config. [SPEAKER_00] With. A lot of.

31:29

SPEAKER_00

Custom rules.

31:29

SPEAKER_01

And those are arbitrary. For example. [SPEAKER_00]

31:29

SPEAKER_00

I don't want the model to do.

31:29

SPEAKER_01

An explicit type assertion on things. I want the model to use schema.

31:29

SPEAKER_03

To check for the shape.

31:29

SPEAKER_01

I have rules prohibiting the usage of. X.

31:29

SPEAKER_03

As.

31:29

SPEAKER_01

Z. I have rules prohibiting the usage of any. Of unknown. Basically. I'm trying to avoid. The model to do dumb stuff. That I realized it was doing. In my code.

31:29

SPEAKER_00

Any is easy to be saved. But type assertions. I think.

31:29

SPEAKER_01

Like you can't. Tell TypeScript config to. [SPEAKER_00] No. Yeah. The same for unknown. And the fine thing is. Initially I banned unknown.

31:35

Because I wanted the model.

32:03

To not do. As unknown.

32:37

As X. It found.

32:45

That never.

32:55

Is a bottom type. So you can do. As never.

33:04

As X. [SPEAKER_01] But type assertions. [SPEAKER_01] I think you can't tell TypeScript config to.

33:17

[SPEAKER_00] No. Yeah. The same for unknown. And the fine thing is initially I banned unknown because I wanted the model to not do as unknown as X. It found that never is a bottom type. So you can do as never as X. Okay. Then I'm gonna ban as. And now it's doing better. Okay. Let's see what it created. Okay. This is sure use ban. Okay. Available project commands. That's fine. Test watch. Hmm. This is gonna create issues. I already know because the model is gonna try to run this and get stuck. Same with dev servers. Fact reference repositories. Good. Look at the for repository specific guidance. Okay. That's enough. [SPEAKER_02]

33:27

[SPEAKER_00] Of a start. Mention in the agents.md. You should never ever try to run commands in watch mode. For example, you are not allowed to run or a dev server. Otherwise it's gonna try to run the dev server as the first thing and get stuck. Okay. What I like about OpenAI models is that they are way more concise compared to Anthropic models. The same task with Opus would have probably wrote 200 lines of agents.md. But that's good. It's enough. It's enough as a start. So we are back to square zero. We said we want to create an HTTP API. I know nothing about Effect. So I would like to create an HTTP API that should have OpenAPI documentation and type-safe client generated by default. Explore the Effect repo for patterns on how to do this. Save your research into patterns HTTP API dot md. Ask me any question you need. Again, I'm starting from the perspective that I have no idea how to do this in Effect. Do you have to use the plan mode in OpenCode or not doing it? No. I find plan mode to be the issue with plan mode is that the model has crippled access to tools. So it cannot easily do the same things that it does outside of plan mode. So not. I don't make heavy usage of it. I usually do what's called spec-driven development. In the sense that the first task I do with the model is I discuss with the model how to create a spec for something. Then the spec is persisted as a markdown file, which is effectively my plan. And I tell the model then to implement. Usually the second step I do in a RALF loop. Because you've seen I already restarted OpenCode a few times to clean up the context window. Doing this manually is boring and you usually end up reusing the same context window for multiple things. And it's going to just de-optimize the model at some point.

33:33

[SPEAKER_00] Because the context window is limited, you're going to push a lot of information in. And the earlier information is going to confuse the model for the later information. So I use a very simple bash script that tells the model pick up a small task. Implement the small task. And then exit. And I run that in a loop. It's funny how with AI many times less is more. You can have very complex architectures around context management and so on and so forth. At the end the dumbest thing ever ends up working better. And we are doing research by ourselves. And it looks like there's actually very good margins of improvement by reducing the number of tools that the model has access to.

33:42

[SPEAKER_01] For example we have been experimenting with a coding agent that has a single tool call which is called execute. And it can execute arbitrary TypeScript code, including calling bash through TypeScript. And in that scenario the model doesn't even have access to a patch. It cannot change files directly. It has to write a TypeScript file that changes the code. And then it ends up doing TypeScript transformers, AST base transformations. It's fantastic how you reduce the things that the model can do and it does better. So let's see. Save the research to HTTP API. Good. Main conclusion. For this repo the strongest default Effect partner is to define the shared HTTP API. You're absolutely right. Derive OpenAPI from it. Mount the docs. Okay.

33:46

[SPEAKER_00] OpenAPI generator only when you need generated client artifacted. We don't know. We don't need that. One question before I implement anything further. Do you want the primary pattern here to be shared HTTP API with HTTP API client dot make? No. I am fine with a shared HTTP API. I don't need a committed client in the repo itself. Let's see what it did here. For this workshop repo the best part. Okay. [SPEAKER_01] You're absolutely right. [SPEAKER_01] Derive OpenAPI from it. [SPEAKER_01] Mount the docs. [SPEAKER_01] Okay. [SPEAKER_00] OpenAPI generator only when you need generated client artifacts.

34:04

[SPEAKER_03] We don't know. [SPEAKER_00] We don't need that.

34:12

[SPEAKER_00] One question before I implement anything further. [SPEAKER_00] Do you want the primary pattern here to be shared HTTP API with HTTP API client dot make?

34:23

[SPEAKER_00] No. [SPEAKER_00] I am fine with a shared HTTP API. [SPEAKER_00] I don't need a committed client in the repo itself. [SPEAKER_00] Let's see what it did here.

34:37

[SPEAKER_00] For this workshop repo the best part. [SPEAKER_00] Okay. [SPEAKER_00] This give you relevant upstream files. [SPEAKER_00] Good. [SPEAKER_00] Ok. [SPEAKER_00] Ok.

35:07

[SPEAKER_00] Ok.

35:18

[SPEAKER_01] Ok.

35:27

[SPEAKER_00] Ok. [SPEAKER_01] Ok. [SPEAKER_00] Ok. Ok.

35:59

Ok.

36:04

Ok. But this is just generic patterns that we're going to use as reference. So list the files in patterns in the agents.md so the agent has context of their existence. Model does not care about grammar.

36:19

SPEAKER_01

And I feel like many people raise the point that a model is not good at something if it doesn't do well by default. I don't think there's anything more wrong with that statement. The model is good when it can operate a large-scale codebase using patterns and it doesn't fail at scale.

36:31

SPEAKER_03

The zero to one problem is not really...

36:32

SPEAKER_00

It's a problem for the first ten days or ten hours depending on what you're building. And us programmers, if our job is not to write code, our job should be to set up the repositories in ways that the models can act good on it. So what I'm doing now is most of what I do when I operate a code engagement at scale in a codebase, even if the codebase has no concept of AI. If I start in a project that is brownfield, codebase existing from five to ten years, no context set up, [SPEAKER_01] the first thing I do is let the model explore the code, clone the main libraries that are used.

37:00

SPEAKER_00

If you're using a framework like TanStack or so on and so forth, clone the code of TanStack router. If you're using Svelte, clone the codebase of Svelte. Ask the model to generate best practice files and so on and so forth. Once you have all of it, the model is going to be much more efficient.

37:11

SPEAKER_03

So now that we have a little bit of context on HTTP APIs, we can start implementing one. I do want to check something quickly because I'm using Bun and I'm using Vtest. There's a Vtest run. Does Vtest run actually use Bun as the runtime or does it use Node?

37:18

SPEAKER_00

Because if I recall there was a flag that I had to pass to Vtest to let it use Bun. And I don't want our test setup to differ from our... What is it doing? No. Add to Vtest that it should ignore anything in repos. It was running the effect tests that it found. There was no Vtest config whatsoever. Good. Add to the test something that uses a Bun API.

37:44

SPEAKER_01

[SPEAKER_00] I feel like I did it here, so I should have...

37:50

SPEAKER_01

Test Vtest one. [SPEAKER_00] No, okay. Was I using Node? Probably. [SPEAKER_00] Ok. Ok. Ok. Ok. Ok. [SPEAKER_00] Ok. Ok. Ok. Because now it did one of the classical mistakes. It had to make the test pass. It changed the test to make it pass. Ok. Ok. Ok. Ok.

38:42

SPEAKER_01

Ok.

38:43

SPEAKER_02

[SPEAKER_01] Ok.

38:44

SPEAKER_01

[SPEAKER_00] Ok.

38:47

SPEAKER_00

Ok. Ok. Ok. [SPEAKER_01] Ok. [SPEAKER_01] Ok.

38:56

SPEAKER_01

[SPEAKER_00] Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

39:23

SPEAKER_01

Ok. Ok. Ok. Ok. Ok. Ok.

39:30

SPEAKER_00

Ok. Ok. Ok.

39:36

SPEAKER_00

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

39:58

SPEAKER_00

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

40:25

SPEAKER_00

Ok. Ok. Ok. Ok. Ok. Ok.

40:33

SPEAKER_02

Ok. Ok. Ok. Ok. Ok. Ok.

40:33

SPEAKER_00

Ok.

40:33

SPEAKER_02

Ok.

40:33

SPEAKER_00

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

41:01

SPEAKER_03

Ok.

41:02

SPEAKER_00

Ok. Ok. Ok.

41:08

SPEAKER_00

Ok. Ok.

41:18

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

42:33

SPEAKER_01

Ok. Ok. Ok.

42:39

SPEAKER_01

Ok.

42:49

SPEAKER_01

Ok.

42:58

SPEAKER_01

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

44:12

SPEAKER_00

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

45:11

SPEAKER_01

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

46:13

SPEAKER_01

Ok.

46:18

SPEAKER_00

Ok.

46:23

SPEAKER_03

Ok.

46:24

SPEAKER_00

Ok. Ok. Ok. Ok. Ok. Ok.

46:59

SPEAKER_00

Ok. Ok. Ok. Ok. Ok.

47:18

SPEAKER_00

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

47:26

SPEAKER_00

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

47:29

SPEAKER_01

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

47:29

SPEAKER_00

Ok.

47:30

SPEAKER_01

Ok.

47:30

SPEAKER_00

Ok.

47:31

SPEAKER_01

Ok. Ok. Ok. Ok. Ok. Ok. Ok.

47:38

Ok. Ok. Ok. Ok.

48:04

Ok.

48:09

Ok. Ok. Ok. Ok. Ok. Ok. Ok.

48:47

SPEAKER_01

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

50:01

Ok. Ok. Ok.

50:27

Ok. Ok.

50:46

Ok. Ok. Ok. Ok.

51:11

Ok. Ok.

51:29

Ok.

51:41

SPEAKER_00

Ok.

51:50

SPEAKER_00

Ok. Ok. Ok. Ok.

51:59

SPEAKER_00

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

52:06

SPEAKER_01

Ok.

52:10

SPEAKER_01

Ok.

52:12

SPEAKER_00

Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok. Ok.

52:13

SPEAKER_01

Ok.

52:13

SPEAKER_00

Ok. Ok.

52:14

SPEAKER_01

Ok. HTTP API implementation. So we want to implement an HTTP API following the

52:14

SPEAKER_00

patterns at... pattern stitch in HTTP API. We want the API to expose a to-do functionality where you can one create to-dos description title description

52:14

SPEAKER_01

to update to-dos, change title, etc. We flag a to-do as done or not. For list to-dos. I should have done something else.

52:14

SPEAKER_00

Discuss the plan with me and create plans to-do API.md.

52:14

SPEAKER_01

So here I'm telling the LLM to read the pattern file that we created before

52:14

SPEAKER_00

where it's going to gather generic knowledge about the effect ways of doing things. It still has access to the original code base of Effect if it wants to. But now I'm creating a specific plan to implement the API that I would like to implement. Ok. Ok. Ok. Ok.

52:23

Ok. Ok. Ok. Ok.

52:54

Ok.

53:07

SPEAKER_01

Ok. Ok. Ok. Ok. [SPEAKER_00] Ok. [SPEAKER_01] Ok. [SPEAKER_01] Ok. [SPEAKER_01] Ok. Ok. [SPEAKER_01] Ok. A to-do functionality where you can create to-dos with title and description. Update to-dos, change title, etc. Flag a to-do as done or not. List to-dos. I should have done something else. Discuss the plan with me and create plans to-do API.md. So here I'm telling the LLM to read the pattern file that we created before where it's going to gather generic knowledge about the effective ways of doing things. It still has access to the original code base of Effect if it wants to. But now I'm creating a specific plan to implement the API that I would like to implement.

53:10

Drafting a plan, okay. You want to do that, that's fine. Initial storage strategy, let's do something different. For storage, use Effect SQL and SQLite store. Explore the Effect repo for how to do that and create patterns.sql.md. I realize we need a persistent strategy and I don't have a persistent strategy. I know that Effect has some SQL thing and again I'm using the same process where I first generate some patterns for it. And this is also useful because you may want to use something from Effect but you may not want to use everything from Effect. So if we were to push all the patterns in your repository by default you would end up using everything from Effect even if you don't want to. This is self-select so you can pick and choose whatever you want to use. Especially in brownfield projects this is very important because you don't want to refactor everything you already have. For example, here I could have picked Result to do the persistence just as well.

53:10

What are the efforts of potentially using the data is something for patterns and other usable components in relation to maybe so that you can use and start a project immediately? Most likely we are going to develop some kind of CLI where you can prefetch some patterns that are already available and still let you pick and choose. And we also want to automate this process of exploring something, creating patterns out of it. Because the patterns that we have as best practices might not exactly fit your needs. So you would still maybe update them as a second step. Also the model I use may not be as good as the one that we use but it also happens. For example, the PRs that are some of the top of the law, you go and you read the code. And the code it generates, all of the documentation everything it generates, it's not as good as the one that we use.

53:10

I would also ask what is the framework library authors are providing. This kind of pattern is a solution but officially distributed by the package, collocated. I feel it's generally a good idea but there are some caveats to that. For example, even the agents.md standard is not a standard because the way you prompt Claude and the way you prompt GPT is different. For example, you've noticed I never wrote anything in uppercase. If I were using Claude I would write a lot of stuff in uppercase. The reason is GPT gets scared if you scream at them. And if you scream at it, it's going to de-optimize and then be passive and agree on everything. That is not what you want. With Claude, if you scream at it, it's going to pay attention to that specific sentence. So that comes also in these shared patterns. I feel the patterns should be generated with the model you use versus being off the shelf.

53:10

Now we can do that for the top three frontier models. All the GPT family is very similar. 5.3, 5.4, 5.2. There are not so many differences. Opus, Sonnet and Haiku are also very similar. So ideally we can have the CLI where it says which model do you use? Okay, I'm going to optimize the context for these versus the context for that. And it's annoying because you would obviously want to have a standard. I would also maintain it as an author of the episode.

53:10

Yes, yes. It's very painful to maintain this stuff. Maybe you can keep it as a derivative so that you have the curated factors for certain things in a non-specific way. And then by each model you turn that into useful factors. So our approach is to make the code as good and self-explanatory with examples and everything that any model you use can generate those. And then the CLI would generate them on the spot for the model you use. That's one approach. It may fail and in six months we provide patterns for everything and just tell you please use either one or two.

53:10

SPEAKER_01

[SPEAKER_01] Another very interesting argument is fine-tune an open source model to use Effect patterns by default. We thought of that. [SPEAKER_01] So let's see. [SPEAKER_01] Okay, let's see. [SPEAKER_01] Update. [SPEAKER_00] VHttp. No. If you want the next step for me to update, yes. Do that. This is the annoying part of GPT models. They are going to ask constantly for input from you to continue. Opus would have just done it. [SPEAKER_03] But sometimes it doesn't work and you have to do it three times after that your session limit.

53:11

SPEAKER_01

[SPEAKER_00] That's why I use GPT 5.4. Well, I'd like some sort of fusion and an inbred fusion of Anthropic models and OpenAI models so that it doesn't ask me all the time. Because GPT usually, especially in complex tasks, takes its time but at the end the output is good. [SPEAKER_02] With Opus is right. Sometimes it likes to take these shortcuts and the fine thing is if you let one slip, it's going to repeat. Like if you let one slip in your code base and if you have Opus, it's going to do as any all the time.

53:11

[SPEAKER_00] It's like, oh, I can do this. Let me do that for everything. I need this to compile. Let's remove the code. Yes. That's why in this project and in accountability I was using Opus and I have a link file of thousands of lines of code to prohibit any shortcut. I can start implementing this next. [SPEAKER_00] Yes, please. I feel we've spent enough time. Let's see what it does. See, it's correctly looking up in the Effect repo in the AI docs for ideas. It's most likely it's going to take a little bit, which is positive. [SPEAKER_00] Do you use anything for schemas? [SPEAKER_02] Kind of.

53:11

[SPEAKER_00] In some projects, it was using schema by default and I didn't need a lot of back pressure for it. [SPEAKER_03] Sometimes, yes, one example is the rule in... In accountability I have this yes lint rule. [SPEAKER_00] For example, SQL. Custom yes lint rule to ban SQL type because it would write an SQL query. It would write an interface and it would just... This is the exact same thing as casting. And I had to ban this pattern fully. It's using type parameters with SQL template, either provides no runtime validation, use SQL schema, find one. [SPEAKER_02] Kind of.

53:11

SPEAKER_01

[SPEAKER_00] In some projects, it was using schema by default and I didn't need a lot of back pressure for it. [SPEAKER_03] Sometimes, yes, one example is the rule in accountability I have this yes lint rule. [SPEAKER_00] For example, SQL. [SPEAKER_00] Custom yes lint rule to ban SQL type because it would write an SQL query. It would write an interface and it would just be the exact same thing as casting. And I had to ban this pattern fully. It's using type parameters with SQL template, either provides no runtime validation, use SQL schema, find one. [SPEAKER_02]

53:11

[SPEAKER_01] And you see that the rule ends up suggesting to use SQL schema. So I'm more or less just watching what the model produces and if there's something I don't like, I end up writing lint rules to prohibit that specific pattern. For example, in schema, many times it would have a user ID as a string and then it would have another ID as a string and you would have no type safety whatsoever and the code would try to pass one into the other. So I would force all identifiers to be branded types. And I would then prohibit the usage of type casting because otherwise it would do, this requires a user ID, let me do as user ID.

53:11

SPEAKER_01

[SPEAKER_00] And that's pointless. You should validate the data. So I would ban the usage of as and force them to use constructors. So instead of doing 100 as user ID, user ID.make. [SPEAKER_00] Or prohibit usage of constructors in places where you should do validation. [SPEAKER_02] For example, one case that I found was it would do the API layer as plain strings and then use constructors inside the handler to create the objects, defeating the purpose. Then I would write rules for the model to write validation directly in the schemas so that I was saying if you use a constructor inside the handler, most likely you're wrong.

53:11

SPEAKER_00

You should improve the starting schema to provide validation at the edge. It's babysitting a junior developer with a knife running through the kitchen instead of a kid running through the kitchen with the knife. But this is still going. So I think. You said that you usually do both Codex and Codex. How do you decide when I should use Codex and when I should use Codex? Both models are exceptional. Sometimes one model drives you nuts and you try the other. There's not much of a rule. Lately, I tend to use more OpenAI models because I don't really like to be restricted on the harnesses that I can use. [SPEAKER_01] What do you do hard?

53:11

[SPEAKER_00] I don't really like to use the CLI itself. I want to use OpenCode. I want to use my own TypeScript files that interact with the AI SDK natively and I'm prohibited from doing that from Anthropic. So up until a few months ago when this was allowed, I would use mostly Opus. When they enforced their policies against OpenCode, I switched to OpenAI models. And now I'm most of the time just using OpenAI models. There are some small edge cases. For example, when you do UI, Opus is much better than Codex.

53:11

SPEAKER_01

[SPEAKER_03] There are some specific things where one is clearly better than the other. But for most of the tasks, they are the same. I just had some experience, for example, where GPT thought for half a day on a bug that I had and went nowhere and Opus one-shotted the solution. But I had the opposite experience too. [SPEAKER_00] So it's very hard to know which one is which. You could fall back, if the data is already too much time to include and you could dramatically, right? [SPEAKER_01] You could.

53:11

[SPEAKER_00] You definitely could. Okay, let's see what is this creating. Okay, it created an SQL client. The layer looks correct. It has migrations. It decided to inline the migrations. Okay, that's a valid choice. It correctly provided the SQL live layer to the migration layers. This feels duplicated. There is clear duplication between these two. I used it for this stuff. It creates multiple things that are doing the same thing and it's not importing it to another. It's also a way to fight with the slogan. Sometimes you defactor and it leaves one code in place and it's never exported, never used in the same file. And it catches up quite well. Okay, good to know. We are in our experimentation, another thing we are doing is we are using semantic code search.

53:11

[SPEAKER_03] Because we've noticed that a lot of times the model re-implements the same features because it doesn't find it. And with semantic code search it finds it.

53:15

[SPEAKER_00] Well, okay. There's a duplication here. I'll probably tell it that there is a duplication at some point. I want to check the API. Exactly. You see it's using plain strings for identifiers. So one of the future things that we might want to do is to tell it to use branded stuff. Okay. To do not found it added a schema annotation to flag that to do not found should be a 404. This looks decent. I don't understand why it sometimes creates structs instead of classes. I personally prefer to use classes. So I would in the future either create a best practice to prefer classes or depending on how strict I want, create a lint rule to prohibit usage of schema.struct in specific files and stuff. For now it's obviously fine. It doesn't need to be. Is it not something that we started asking? Is it not something that we started asking? Is it not something that we started asking?

53:19

[SPEAKER_03] Not sure. There might be?

53:45

[SPEAKER_00] But it's not flagging anything here so. And the LSP is on. Lint all the files. Lint with what? We don't have a linter in place. Good point. We also do not have a formatter in place. Let's ignore for now. Let's see. [SPEAKER_01] Let's see. Okay. [SPEAKER_00] Client with a base URL. That's good. We have the live handler. [SPEAKER_01] We have the live handler.

54:38

[SPEAKER_02] Server.

54:49

[SPEAKER_03] There might be? [SPEAKER_00] But it's not flagging anything here so...

55:12

[SPEAKER_00] And the LSP is on. [SPEAKER_00] Lint all the files. [SPEAKER_00] Lint with what? [SPEAKER_00] We don't have a linter in place. [SPEAKER_00] Good point. [SPEAKER_00] We also do not have a formatter in place. [SPEAKER_00] Let's ignore for now. [SPEAKER_00] Let's see.

56:03

[SPEAKER_01] Let's see.

56:11

[SPEAKER_01] Okay. [SPEAKER_00] Client with a base URL.

56:20

[SPEAKER_00] That's good. [SPEAKER_00] We have the live handler. [SPEAKER_01] We have the live handler. [SPEAKER_02] Server. [SPEAKER_02] Index is just exporting everything. [SPEAKER_02] The index.ts should probably run the server instead of exporting everything. [SPEAKER_02] Do that condition checking that the file is main so it doesn't run when you import the file. [SPEAKER_02] It also created some tests. [SPEAKER_02] Do you have some global agents at the file? [SPEAKER_02] No. [SPEAKER_00] No. [SPEAKER_00] What is it doing here? [SPEAKER_00] It created an arbitrary with HTTP to run an effect. [SPEAKER_00] Make test HTTP live. [SPEAKER_00] Okay.

56:20

SPEAKER_00

[SPEAKER_00] It's one way. [SPEAKER_00] Do the test actually pass? [SPEAKER_00] Come on. [SPEAKER_00] Run test. [SPEAKER_00] I'd be surprised. [SPEAKER_00] Wow. [SPEAKER_00] Is there a start command? [SPEAKER_00] Add a start command to start the API server and tell me where to find the Open API docs. [SPEAKER_00] Okay. [SPEAKER_01] It really likes this pattern. [SPEAKER_00] As a future thing, I would probably just tell it to use it.layer instead of using the width repository. [SPEAKER_00] And width thing. [SPEAKER_00] But let's see if at least it works. [SPEAKER_00] Ban run start. [SPEAKER_00] Good. Way of listing to do this.

56:20

SPEAKER_01

[SPEAKER_00] Let's check the Open API. [SPEAKER_03] Good. [SPEAKER_03] There is an Open API. [SPEAKER_00] That's what created. [SPEAKER_00] This looks decent as a first. It shows the schemas properly. Good. Okay. Then let's... Let me... [SPEAKER_00] It did create a database here.

56:20

SPEAKER_00

[SPEAKER_01] Let me maybe git ignore the full DB. [SPEAKER_01] DB to do's dot DB dot all. [SPEAKER_01] To do's dot DB. To do. [SPEAKER_01] You're right. [SPEAKER_01] To do. [SPEAKER_01] You're right. [SPEAKER_01] Yeah. [SPEAKER_01] No longer able to write anything by hand. [SPEAKER_01] Yes.

56:30

[SPEAKER_01] Okay.

56:53

[SPEAKER_01] Let's actually clean up the tests a little bit.

57:03

[SPEAKER_01] So.

57:21

[SPEAKER_02] You see.

57:33

[SPEAKER_02] I'm fooling myself in wanting to use the same session over and over again.

57:38

[SPEAKER_01] That's when rough loops are really useful. We created a lot of mess.

57:51

[SPEAKER_01] You created a lot of mess in tests. [SPEAKER_01] Clean up everything. [SPEAKER_01] This should be the cleanest code you've ever seen. [SPEAKER_01] Not the crappy Python code you've been trained on. [SPEAKER_01] So.

58:44

[SPEAKER_01] Do not use patterns.

58:54

[SPEAKER_01] Simply use git dot layer with layer. [SPEAKER_01] And put utilities in their own folder. [SPEAKER_01] No offense to Python developers of course. [SPEAKER_01] Probably. [SPEAKER_01] Now. [SPEAKER_01] Now I'm winging git. [SPEAKER_00] I'm going to see if it's able to do it. [SPEAKER_00] If it does once it's done.

59:57

[SPEAKER_00] I'm going to create a pattern from it. [SPEAKER_00] But yes. [SPEAKER_01] That would have been a good idea. [SPEAKER_01] Which is why automating the process is very important. [SPEAKER_01] Because we are lazy. [SPEAKER_01] Now I was so lazy that I didn't want to create a pattern for it. [SPEAKER_01] Maybe I'll use test-utils-layers. [SPEAKER_01] Maybe. Maybe. Maybe.

1:01:10

[SPEAKER_01] Maybe. [SPEAKER_01] So can you say it's bad or it's bad or it's bad passing there? [SPEAKER_01] Yeah. [SPEAKER_01] Oh. [SPEAKER_01] The bad pattern. [SPEAKER_01] Yeah. [SPEAKER_01] It basically created a function to provide a layer to an effect. [SPEAKER_01] It built the layer manually. [SPEAKER_01] It wrapped everything in an effect.scoped which is going to close the layer once this is done. [SPEAKER_01] And my guess is that it did this because this pattern is actually used to test some layer internals in the codebase. [SPEAKER_01] But it's completely unnecessary here.

1:02:02

[SPEAKER_01] But if you look at the file, even without knowing details of effect, it stinks. [SPEAKER_01] Something's not right. [SPEAKER_03] Now it cleaned it up. [SPEAKER_03] So when you see something that doesn't look right, usually just ask the model why you did that. [SPEAKER_01] Is there any alternative? [SPEAKER_01] And in this case I knew that to provide a layer in test we should just use it.layer. [SPEAKER_01] So I kind of skipped that. [SPEAKER_00] But in reality I would have, if I didn't notice, I would have discussed with the model that I didn't the repeated thing all over. [SPEAKER_00] And sometimes it's necessary.

1:02:50

SPEAKER_01

Sometimes you're wrong and the model is right. That's the way to do it.

1:03:05

SPEAKER_01

But in this case, it was completely unnecessary. Do we have describe.layer as well? Or do we not do it?

1:03:13

SPEAKER_01

No, I think we have it.describe.

1:03:15

SPEAKER_00

[SPEAKER_01] So you would do it.layer as a top thing. [SPEAKER_01] Pass the layer. [SPEAKER_01] Whatever. [SPEAKER_01] Then in the closure do it.describe. [SPEAKER_01] Could probably also add any .describe as a short. [SPEAKER_01] Models don't care about verbose code. [SPEAKER_01] Why should we make it less verbose?

1:03:36

SPEAKER_03

[SPEAKER_01] Does it do any cleanups? [SPEAKER_01] Yes.

1:03:41

SPEAKER_00

So if we do it in the top level, it will poison the other tests. And the other alternative is that you just. The other alternative is that you just. The other alternative is you provide it.layer at every test.

1:04:05

SPEAKER_02

[SPEAKER_00] The reality is whenever you're using a database, in this case it says SQLite, so the argument is kind of moot, but if I were to use a Postgres in a project where you have hundreds of tests, spinning up a Postgres instance per test is going to make your test runtime two days maybe. So usually what I end up doing is I end up making tests that can run, that do self-cleanup, for example I run a test within a transaction and I roll back the transaction as soon as the test finishes so that they are kind of atomic by the fact that they don't leak that. It would be another pattern that we can tell the model to do. It would be a matter of creating the transaction and the rollback.

1:04:07

SPEAKER_02

[SPEAKER_00] The reality is whenever you're using a database, in this case it says SQLite, so the argument is moot, but if I were to use a Postgres in a project where you have hundreds of tests, spinning up a Postgres instance per test is gonna make your test runtime two days maybe. So usually what I end up doing is I end up making tests that can run, that do self-cleanup. For example, I run a test within a transaction and I roll back the transaction as soon as the test finishes so that they are atomic by the fact that they don't leak. It would be another pattern that we can tell the model to do. It would be a matter of creating the transaction and the rollback. But there's alternatives.

1:04:15

SPEAKER_02

[SPEAKER_01] How does the model know about the effect library and related to the effect 7 or something? Or are you just relying on the model's knowledge about the library? [SPEAKER_03] No. We added the effect code base in a repository folder. We created an agents.md that references the effect repo. And then for the features we wanted to use, we asked the model to create patterns by looking at the repo, investigating how things are done in the repo as general knowledge. In this case we did one for SQL, we did one for API. Now the good point is in this session we have best practices about testing. So let's create patterns slash testing dot md.

1:04:21

SPEAKER_02

[SPEAKER_00] So let's see. It should include all the best practices of testing effect based code. Including usage of it dot layer and here, etc. And also update agents dot md to reference all the patterns in dot patterns. And the next thing that you would do to automate the flow is, for example, open code allows you to create slash commands, and cloud code allows you to do the same. You optimize for slash new pattern, whatever you want. Would you leave your skill, for example, and discover it in this one? You can create skills and tag the skills. Skills are very useful for these kinds of things. I'm against skills in general. For these things they are ideal, but many people think that just by adding a skill you're gonna make the model good at React. You're gonna make the model good at Next.js. The reality is if you put a skill for every single Next.js internal, you're gonna pollute the context and not get anywhere. So skills have a very good use case, which is this kind of use case.

1:04:22

SPEAKER_00

[SPEAKER_03] And I guess they are more general than slash commands. So I tend to do slash commands because I tend to use a single coding agent. But definitely, if you are, for example, in a team where everybody's free to use their own agent, maybe some people use cursor, some people use open code, some people use code code. Skills are a good baseline. Let's see patterns testing. [SPEAKER_01] Use effect-b-test for all effect-based tests. Use it.effect. Use it.layer. Avoid custom wrappers.

1:04:30

SPEAKER_00

[SPEAKER_02] Use that.call.layer.build. This is a very specific rule. Now, a friend of mine told me whenever you read a rule book, a legal rule book, you find those specific rules that are just when you enter a pub and it's, don't do skateboarding on top of. And you ask yourself, why does this rule exist? Because somebody did that. Why does this rule exist? Because the model did that. Why this pattern? Okay. You see relevant files. They're all linked. They need to be maintained as the file references. Yes.

1:04:38

SPEAKER_00

[SPEAKER_02] And there's a friend of mine who's writing a linter plugin that checks for existing references. So when you add code, when you change code, it runs in the CI and says, hey, this reference is broken. So sometimes, instead of relative time, they give the absolute time? Yes. And you see your name? Yes. The full slash home slash whatever. Yeah, yeah. I guess another approach you can use is to actually write tests for your pattern. If you think that's the wrong answer. And then you can keep evaluating them.

1:04:57

SPEAKER_00

[SPEAKER_01] How would you write a test for a pattern? In the problem. You have a pattern in your test. You mean actually write a file? You use the pattern in the test and then evaluate the result. You use the pattern in the test.

1:05:06

SPEAKER_00

I feel that could be a way. Sometimes the code that is inside the patterns is not really executable. I guess it has pros and cons. It's definitely an interesting idea.

1:05:31

SPEAKER_00

[SPEAKER_02] I guess it's a real idea. For example, maybe with an additional tag like ts, execute these to flag which of the patterns you actually want executed or references, which files you want to be referenced. Because sometimes it mentions files as examples. For example, if you write this feature, you use the file called abc and that's not a concrete reference. So you don't want your program to fail because it read that. Maybe you can just have a command using your pattern to actually write code and then you evaluate the output. That's more in the direction of evaluations. So evolves.

1:05:42

SPEAKER_00

Yes, that's at scale. That's very good. I found doing it on a per project basis ends up. Yeah, not on purpose, but if it's more your domain, because you spend more time curating these things that actually write in code or eventually doing code. So maybe I'm saying for the vertical code as you use the code? What we are thinking of doing in the effect repo is, for example, to have evals running once per day and generating reports. So anytime we do library changes or we add more docs, we add more examples, we see exactly if the outputs are better or are worse. Sometimes in evals it's very hard. Even Anthropic a while ago wrote a blog post where the summary of the blog post is we don't really know when code is good or bad because is more terse code better? Depends. Is more verbose code better? Depends. There are some properties where you can say this is definitely better than not, code that type checks is better than code that doesn't. Probably true. But when it comes to style, when it comes to is this file structure better than another file structure and they both convey meaning, you need a human at the end to say yeah I prefer this.

1:05:43

SPEAKER_00

if the outputs are better or are worse. Sometimes in evals it's very hard. Even Anthropic a while ago wrote a blog post where the summary of the blog post is we don't really know when code is good or bad because is more terse code better? Depends. Is more verbose code better? Depends.

1:05:52

SPEAKER_02

[SPEAKER_00] There are some properties where you can say this is definitely better than not, like code that type checks is better than code that doesn't. Probably true.

1:05:57

SPEAKER_00

But when it comes to style, when it comes to is this file structure better than another file structure and they both convey meaning, you need a human at the end to say yeah I prefer this and if you take 100 humans you're going to have an 80-20 split. So we have the same problem now with defining effect patterns because we are running evals and evals are our opinion of what's good and it's not really an absolute truth. Let's put it this way.

1:06:10

SPEAKER_03

[SPEAKER_02] So do you have an LLM that checks for certain patterns? How do you run the evals?

1:06:23

SPEAKER_03

[SPEAKER_00] We have humanly written best practice code. We have generated code and then we have an LLM that matches and says is this too different or not? Give us a score and that's pretty much how you run the evals.

1:06:28

SPEAKER_00

So not a very nice way to run. But we're trying to figure this out because we are thinking of fine-tuning a model on top of effect and for the reinforcement learning part we are going to need to have good evals.

1:06:36

SPEAKER_00

So it's part of what we are researching right now. There's no right or wrong answer. [SPEAKER_02] So if there was all the models we perform the same because everybody would have the same evals, everybody would have the same thing. [SPEAKER_02] But now we have all the patterns for what we want. So I feel we are at the point of saying commit this.

1:06:55

SPEAKER_00

[SPEAKER_02] So I'm going to create a repository and push it so that at least you have access to it. New repository.

1:07:05

SPEAKER_02

[SPEAKER_00] Ok.

1:07:06

SPEAKER_01

[SPEAKER_00] the final repository so hopefully. So we haven't got to the point of doing clustering and workflows. Just sharing a few words about why you would want those aspects in your code. This is a very dumb to-do API. One thing I wanted to add would be authentication and registration. For example when you have a registration your process is usually write something in the database and then send an email or send an email code and wait for confirmation. Anytime you do two unrelated operations there is no transaction between them, no database transaction between them and your server may fail at any random point within your code. So it's very hard to guarantee that the email has actually been sent which is why many times in a registration procedure you see the sentence if the email did not arrive in 30 minutes please retry. You retry for me, why should I retry if I haven't received the email? That's a symptom of a badly designed system. They cannot guarantee that two operations happened. To do that you have various ways.

1:07:17

SPEAKER_01

One way is to implement queues and so on and so forth. The other way is to use something like workflows. You have solutions like temporal, ingest, there's many workflow solution. Effect has one implemented on top of what is called effect cluster where basically you run a cluster of band, node, whatever instances and the system itself guarantees that once a procedure starts it's going to finish. Even if the server crashes it's going to move to a different location. How I would go about it? Same way as I did now. Ask the model to explore the repository, extract the best patterns around how to use effect cluster, how to use effect workflows and just gone from there.

1:07:32

SPEAKER_01

It's very interesting. It's still in the unstable part of effect but it's going to be stable very soon and we think especially with if you do if you integrate AI in your app it's going to be even more important because with AI every process becomes more long-running like LLM takes minutes to answer. There's a lot of things that can go wrong in a minute. If the average response time is 10 milliseconds the server is pretty much never going to fail in that 10 milliseconds. If that 10 milliseconds becomes a minute, yes you're pretty sure that the server is going to fail in that minute at some point.

1:07:52

SPEAKER_01

And usually before the companies that would use workflows were larger scale companies because at scale every edge case happens twice per day, with longer response time even if you have 10 users you're pretty much going to have disruption if your average process takes a minute and you're going to have failure all over the place. Which is why for example Temporal became much more interesting in the past 12 months because everybody is now implementing AI in their own products. They have chatbots, they have any kind of AI driven process. And with the fact you get workflows, you get clustering, you have AI integrations, you have Discord, Slack integrations and so on and so forth. So the system is really composable and the models are pretty decent at it.

1:07:56

SPEAKER_01

We have a working API. I've been speaking for about an hour and a half and I started with zero effect knowledge. It was an empty repository and this is why I wanted to call this workshop just clone the fucking repo.

1:08:08

SPEAKER_00

[SPEAKER_01] So that's pretty much it. If you have any question or anything else I'm happy to discuss with you at a later point. And let's get the next speaker set up. Thank you so much. [SPEAKER_01] Thank you. So I would ban the usage of as and force them to use constructors. So instead of doing 100 as user ID, user ID.make. Or prohibit usage of constructors in places where you should do validation.

1:08:31

SPEAKER_02

For example, one case that I found was it would do the API layer as plain strings and then use constructors inside the handler to create the objects, defeating the purpose. Then I would write rules for the model to write validation directly in the schemas so that I was basically saying if you use a constructor inside the handler, most likely you're wrong.

1:09:02

SPEAKER_00

You should improve the starting schema to provide validation at the edge. It's kind of babysitting a junior developer with a knife running through the kitchen instead of a kid running through the kitchen with the knife. But this is still going. So, I think. So, I know. You said that you usually do both codex and codex. How do you, like, decide when I should use codex and when I should use codex? I... Both models are exceptional. Sometimes one model drives you nuts and you try the other. There's not much of a rule. Lately, I tend to use more OpenAI models because I don't really like to be restricted on the harness that I can use.

1:09:59

SPEAKER_01

What do you do like hard?

1:10:00

SPEAKER_00

I don't really like to use the CLI itself. I want to use OpenCode. I want to use my own TypeScript files that interact with the AISDK natively and I'm prohibited from doing that from Anthropic. So, up until a few months ago when this was allowed, I would use mostly Opus.

1:10:26

SPEAKER_01

When they enforced their policies against OpenCode, I switched to OpenAI models. And now I'm most of the time just using OpenAI models.

1:10:38

SPEAKER_00

There are some small edge cases. For example, when you do UI, Opus is much better than Codex.

1:10:49

So, for... There are some specific things where one is clearly better than the other. But for most of the tasks, they are the same. I just had some experience, for example, where GPT thought for half a day on a bug that I had and went nowhere and Opus one-shotted the solution. But I had the opposite experience too. So, it's very hard to know which one is which. You could fall back, you know, like a stupid one and if the data is already too much time to include and... You could. ... dramatically, right?

1:11:30

SPEAKER_01

You could. You could.

1:11:31

SPEAKER_00

You definitely could. Okay, let's see what is this creating. Okay, it created an SQL client. The layer looks...

1:11:47

SPEAKER_00

...correct.

1:11:51

SPEAKER_00

...has migrations. It decided to inline...

1:12:02

SPEAKER_00

...the migrations. Okay, that's a valid choice.

1:12:15

SPEAKER_00

Okay. It correctly provided the SQL Live layer to the migration layers.

1:12:34

SPEAKER_00

This feels like duplicated.

1:12:38

SPEAKER_00

There is clear duplication between these two. I used... ...for this stuff. It creates multiple things that are doing the same thing and it's not importing it to another... It's also a way to fight with the slogan. Like, sometimes you defactor and it leaves one code in place. ...and it's like never exported, never used in the same file. And it catches up quite well. Okay, good to know.

1:13:12

SPEAKER_03

We are... ...in our experimentation, another thing we are doing is we are using semantic code search. Because we've noticed that a lot of times the model re-implements the same features because it doesn't find it. And with semantic code search it finds it.

1:13:33

SPEAKER_00

Well, okay. Here's... There's a duplication here. I'll probably tell it that there is a duplication at some point. I want to check the API. Exactly. You see it's using plain strings for identifiers. So one of the future things that we might want to do is to tell it to use branded stuff.

1:14:04

SPEAKER_00

Okay. To do not found it added a schema annotation to flag that to do not found should be a 404.

1:14:21

SPEAKER_00

This looks decent. I don't understand why it sometimes creates structs instead of classes. I personally prefer to use classes.

1:14:36

SPEAKER_00

So I would in the future either create a best practice to prefer classes or depending on how strict I want, create a lint rule to prohibit usage of schema.struct in specific files and stuff like that. For now it's obviously... It's fine. It doesn't need to... Is it not something that we started asking? Is it not something that we started asking? Is it not something that we started asking?

1:15:07

SPEAKER_03

Not sure. There might be?

1:15:10

SPEAKER_00

But it's not flagging anything here so... And the lsp is on.

1:15:20

SPEAKER_00

Lint all the files.

1:15:26

SPEAKER_00

Lint with what? We don't have a linter in place.

1:15:31

SPEAKER_00

Good point. We also do not have a formatter in place.

1:15:39

SPEAKER_00

Let's ignore for now. Let's see.

1:15:45

SPEAKER_01

Let's see. Okay.

1:16:00

SPEAKER_00

Client with a base URL. That's good. We have the live handler.

1:16:09

SPEAKER_01

We have the live handler.

1:16:14

SPEAKER_02

Server. Index is just exporting everything.

1:16:24

SPEAKER_02

The index.ts should probably run the server instead of exporting everything. Do that condition checking that the file is main so it doesn't run when you import the file.

1:17:08

SPEAKER_02

It also created some tests.

1:17:12

SPEAKER_02

Do you have some global agents at the file? No.

1:17:17

SPEAKER_00

No.

1:17:24

SPEAKER_00

What is it doing here?

1:17:32

SPEAKER_00

It created an arbitrary with HTTP to run an effect.

1:17:43

SPEAKER_00

Make test HTTP live. Okay. It's one way.

1:18:00

SPEAKER_00

Do the test actually pass? Come on. Run test. I'd be surprised. Wow.

1:18:09

SPEAKER_00

Is there a start command? Add a start command to start the API server and tell me where to find the open API docs.

1:18:40

SPEAKER_00

Okay.

1:18:41

SPEAKER_01

It really likes this pattern.

1:18:44

SPEAKER_00

As a future thing, I would probably just tell it to use it.layer instead of using the width repository. And width thing. But let's see if at least it works. Ban run start.

1:19:15

SPEAKER_00

Good. Way of listing to do this.

1:19:31

SPEAKER_00

Let's check the open API.

1:19:34

SPEAKER_03

Good. There is an open API.

1:19:37

SPEAKER_00

That's what created.

1:19:41

SPEAKER_00

This looks decent as a first.

1:19:48

SPEAKER_01

It shows the schemas properly. Good. Okay. Then let's... Let me...

1:20:00

SPEAKER_00

It did create a database here.

1:20:06

SPEAKER_01

Let me maybe git ignore the full DB. DB to do's dot DB dot all. To do's dot DB.

1:20:25

SPEAKER_01

To do. You're right.

1:20:35

SPEAKER_01

To do. You're right. Yeah. No longer able to write anything by hand.

1:20:51

SPEAKER_01

Yes. Okay. Let's actually clean up the tests a little bit.

1:21:06

SPEAKER_01

So. Clean.

1:21:10

SPEAKER_02

You see. I'm fooling myself in wanting to use the same session over and over again.

1:21:18

SPEAKER_01

That's when rough loops are really useful. We created a lot of mess.

1:21:30

SPEAKER_01

You created a lot of mess in tests. Clean up everything. This should be the cleanest code you've ever seen. Not like the crappy Python code you've been trained on. So. Do not use patterns like...

1:22:08

SPEAKER_01

Simply use git dot layer with layer.

1:22:19

SPEAKER_01

And put utilities in their own folder. No offense to Python developers of course.

1:22:35

SPEAKER_01

Probably. Now. Now I'm winging git.

1:22:39

SPEAKER_00

I'm gonna see if it's able to do it. Uh. If it does once it's done. I'm gonna create a pattern from it. But yes.

1:22:47

SPEAKER_01

That would have been a good idea. Which is why automating the process is very important. Because we are lazy. Like now I was so lazy that I didn't wanna create a pattern for it.

1:23:05

SPEAKER_01

Maybe I'll use test-utils-layers. Maybe. Maybe. Maybe. Maybe. So can you say it's bad or it's bad or it's bad passing there? Yeah. Oh. The bad pattern. Yeah. It basically created a function to provide a layer to an effect. It built the layer manually. It wrapped everything in an effect.scoped which is gonna close the layer once this is done. And my guess is that it did this because this pattern is actually used to test some layer internals in the codebase. But it's completely unnecessary here.

1:23:54

SPEAKER_01

But if you look at the file, even without knowing details of effect, it stinks.

1:24:07

SPEAKER_01

Something's not right.

1:24:09

SPEAKER_03

Now it cleaned it up. So when you see something that doesn't look right, usually just ask the model why you did that.

1:24:19

SPEAKER_01

Is there any alternative? And in this case I knew that to provide a layer in test we should just use it.layer. So I kind of skipped that.

1:24:31

SPEAKER_00

But in reality I would have, if I didn't notice, I would have discussed with the model that I didn't like to see that repeated thing all over. And sometimes it's necessary.

1:24:46

SPEAKER_01

Sometimes you're wrong and the model is right. That's the way to do it. But in this case, it was completely unnecessary. Do we have describe.layer as well? Or do we not do it?

1:25:03

SPEAKER_01

No, I think we have it.describe. So you would do it.layer as a top thing. Pass the layer. Like whatever.

1:25:22

SPEAKER_01

Then in the closure do it.describe.

1:25:29

SPEAKER_01

Could probably also add any .describe as a short.

1:25:36

SPEAKER_01

Models don't care about verbose code. Why should we make it less verbose? Does it do any cleanups? Yes.

1:25:47

SPEAKER_00

So if we do it in the top level, it will poison the other tests.

1:26:05

SPEAKER_00

And the other alternative is that you just. The other alternative is that you just. The other alternative is you provide it.layer at every test. The reality is whenever you're using a database, in this case it says Qlite, so the argument is kind of moot, but if I were to use a Postgres in a project where you have hundreds of tests, spinning up a Postgres instance per test is gonna make your test runtime two days maybe. So usually what I end up doing is I end up making tests that can run, that do self-cleanup, like for example I run a test within a transaction and I roll back the transaction as soon as the test finishes so

1:27:00

SPEAKER_00

that they are kind of atomic by the fact that they don't leak that. It would be another pattern that we can tell the model to do. It would be a matter of creating the transaction and the rollback. But there's alternatives.

1:27:21

And... How does the model know about the effect library and related to the effect 7 or something like this? Or are you just relying on the model's knowledge about the library? No. We added the effect code base in a repository folder. We created an agents.md that references the effect repo. And then for the features we wanted to use, we asked the model to create patterns patterns by looking at the repo, investigating how things are done in the repo as kind of general knowledge. In this case we did one for SQL, we did one for API. Now the good point is in this session we have best practices about testing. So let's create patterns slash testing dot md. So let's see...

1:28:21

SPEAKER_00

It should include all the best practices of testing effect based code. Including usage of it dot layer...

1:28:41

SPEAKER_00

Here, etc.

1:28:46

SPEAKER_00

And also update... We're gonna queue that... Agents dot md to reference all the patterns in dot patterns. And the next thing that you would do to automate the flow is, for example, open code allows you to create slash commands, and cloud code allows you to do the same. You optimize for slash, new pattern, whatever you want, and... Would you leave your skill, for example, and discover it in this one? You can create skills and tag the skills. Skills are very useful for these kind of things.

1:29:34

SPEAKER_00

I'm kind of against skills in general. For these things they are ideal, but many people think that just by adding a skill you're gonna make the model good at React. You're gonna make the model good at Next.js. The reality is if you put a skill for every single Next.js internal, you're gonna pollute the context and not get anywhere. So skills have a very good use case, which is this kind of use case.

1:30:03

SPEAKER_03

And I guess they are more general than slash commands. So I tend to do slash commands because I tend to use a single coding agent.

1:30:11

SPEAKER_00

But definitely, if you are, for example, in a team where everybody's free to use their own agent, maybe some people use cursor, some people use open code, some people use code code. Skills are a good baseline.

1:30:29

SPEAKER_00

Let's see patterns testing.

1:30:33

SPEAKER_01

Use effect-b-test for all effect-based tests.

1:30:37

SPEAKER_00

Use it.effect. Use it.layer. Avoid custom wrappers.

1:30:45

SPEAKER_02

Use that.call.layer.build. This is a very specific rule. Now, a friend of mine told me whenever you read a rule book, a legal rule book,

1:30:56

SPEAKER_00

you find those specific rules that are just like when you enter a pub

1:31:01

SPEAKER_03

and it's like, don't do skateboarding on top of...

1:31:04

SPEAKER_00

And you ask yourself, why does this rule exist? Because somebody did that. Why does this rule exist? Because the model did that.

1:31:14

Why this pattern? Okay. You see relevant files. They're all linked. They need to be maintained as the file references. Yes. And there's a friend of mine who's writing a linter plugin that checks for existing references. So when you add... When you change code, it runs in the CI and says, hey, this reference is broken. So sometimes, instead of relative time, they give the absolute time?

1:31:54

SPEAKER_00

Yes. And you see your name? Yes. The full slash home slash whatever. Yeah, yeah. I guess another approach you can use is to actually write tests for your pattern. If you think that's the wrong answer. And then you can keep evaluating them. How would you write a test for a pattern?

1:32:17

SPEAKER_01

In the problem. Like, you have a pattern in your test. You mean actually write a file? You use the pattern in the test and then evaluate as a result. You use the pattern in the test. You use the pattern in the test.

1:32:35

SPEAKER_00

I feel like that could be a way. Sometimes the code that is inside the patterns is not really executable. I guess it has pros and cons. It's definitely an interesting idea.

1:32:51

SPEAKER_02

I guess it's a real idea.

1:32:51

SPEAKER_00

For example, maybe with an additional tag like ts, execute these to flag which of the patterns you actually want executed or like references, which files you want to be referenced. Because sometimes it mentions files as examples. For example, if you write this feature, you use the file called abc and that's not a concrete reference. So you don't want your program to fail because it read that. Maybe you can just have a command using your pattern to actually write code and then you evaluate the output. That's more in the direction of evaluations. So evolves.

1:33:48

SPEAKER_00

Yes, that's at scale. That's very good. I found doing it on a per project basis ends up... Yeah, not on purpose, but if it's more like your domain, because you spend more time curating these things that actually write in code or eventually doing code. So maybe I'm saying for the vertical code as you use the code? What we are thinking of doing in the effect repo is, for example, to have evils running once per day and generating reports. So anytime we do library changes or we add more docs, we add more examples, we see exactly if the outputs are better or are worse. Sometimes in evils it's very hard. Like even Anthropic a while ago wrote a blog post

1:34:42

SPEAKER_00

where the summary of the blog post is we don't really know when code is good or bad because is more Terrence code better? Depends. Is more verbose code better? Depends. There are some properties where you can say this is definitely better than not, like code that type check is better than code that doesn't. Probably true. But when it comes to style, when it comes to like is this file structure better than another file structure and they both convey meaning, you kind of need a human at the end to say yeah I prefer this and if you take 100 humans you're gonna have an 80-20 split. So we have the same problem now with defining effect patterns because we are running evils

1:35:38

SPEAKER_00

and evils are kind of our opinion of what's good and it's not really an absolute truth. Let's put it this way.

1:35:48

SPEAKER_02

So do you have an LLM that checks for certain patterns? How do you run the evils?

1:35:56

SPEAKER_00

We have humanly written best practice code. We have generated code and then we have an LLM that matches and says is this too different or not? Give us a score and that's pretty much how you run the evils. So not a very nice way to run. But we're trying to figure this out because we are thinking of fine-tuning a model on top of effect and for the reinforcement learning part we are gonna need to have good evils. So it's part of what we are researching right now. There's no right or wrong answer.

1:36:40

SPEAKER_02

So if there was all the models we perform the same because everybody would have the same evils, everybody would have the same thing. But now we have all the patterns for what we want. So I feel like we are at the point of saying commit this.

1:37:01

SPEAKER_02

So I'm going to create a repository and push it so that at least you have access to it.

1:37:12

SPEAKER_02

Gosh, I'm too weak.

1:37:16

SPEAKER_00

New repository.

1:37:29

SPEAKER_00

Ok. Ok. Ok. Ok.

1:37:39

SPEAKER_00

!

1:37:44

SPEAKER_00

Ok. !

1:37:52

SPEAKER_00

Ok. the final repository so hopefully. So we haven't got to the point of doing clustering and workflows. Just sharing a few words about why you would want those aspects in your code. This is a very dumb to-do API. One thing I wanted to add would be authentication and registration. For example when you have a registration your process is usually write something in the database and then send an email or send an email code and wait for confirmation. Anytime you do two unrelated operation there is no transaction between them, no database transaction between them and your server may fail at any random point within your

1:38:49

SPEAKER_00

code. So it's very hard to guarantee that the email has actually been sent which is why many times in a registration procedure you see the sentence if the email did not arrive in 30 minutes please retry. You retry for me, why should I retry if I haven't received the email? That's a symptom of a badly designed system. They cannot guarantee that two operations happened. To do that you have various ways.

1:39:22

SPEAKER_01

One way is to implement queues and so on and so forth. The other way is to use something like workflows. You have solutions like temporal, ingest, there's many workflow solution. Effect has one implemented on top of what is called effect cluster where basically you run a cluster of band, node, whatever instances and the system itself guarantees that once a procedure starts it's gonna finish. Even if the server crashes it's gonna move to a different location. How I would go about it? Same way as I did now. Ask the model to explore the repository, extract the best the patterns around how to use effect cluster, how to use effect workflows and just gone from there.

1:40:15

It's very interesting. It's still in the unstable part of effect but it's gonna be stable very soon and we think especially with if you do if you integrate AI in your app it's gonna be even more important because with AI every process becomes more long-running like LLM takes minutes to answer. There's a lot of things that can go wrong in a minute. If the average response time is 10 milliseconds the server is pretty much never gonna fail in that 10 milliseconds. If that 10 milliseconds becomes a minute, yes you're pretty sure that the server is gonna fail in that minute at some point. And usually before the companies that would use workflows were larger scale companies

1:41:09

because at scale every edge case happens twice per day, with longer response time time even if you have 10 users you're pretty much gonna have disruption if your average process takes a minute and you're gonna have failure all over the place. Which is why for example Temporal became much more interesting in the past 12 months because everybody is now implementing AI in their own products. They have chatbots, they have any kind of AI AI driven process. And with the fact you get workflows, you get clustering, you have AI integrations, you have discord, Slack integrations and so on and so forth. So the system is really composable and the models are pretty decent at it.

1:42:04

We have a working API. I've been speaking for about an hour and a half and I started with zero effect knowledge. It was an empty repository and this is why I wanted to call this workshop just clone the fucking repo. So that's pretty much it. If you have any question or anything else I'm happy to discuss with you at a later point. And let's get the next speaker set up. Thank you so much. Thank you.

Reading tools

Type to find a passage

Appearance
Ask this transcript

Notebook

Export Markdown

Add a note