AI Engineer

Why your agents need decision traces, not just documents — Zach Blumenfeld, Neo4j

957 summary words 4 min summary Watch video

Start with the signal

4 min read

Summary

30-second take

Neo4j's Zach Blumenfeld argues agents need "context graphs"—not just knowledge bases, but structured decision traces showing why past decisions were made. The key innovation: storing causal reasoning chains (events + context + precedents) as graph nodes, then using graph embeddings to retrieve structurally and semantically similar past decisions. His demo shows a financial fraud agent rejecting a transaction by citing specific past precedents with similar decision patterns. Neo4j just shipped create-context-graph, a CLI boilerplate tool (like create-react-app) that scaffolds agent apps with memory, reasoning traces, and graph retrieval out of the box. This is early-stage but targets a real agent limitation: decisions without precedent or explainability.

Key takes

  • Context graphs = knowledge base + decision memory + reasoning traces: Agents need not just facts (entities, transactions) but why decisions were made—stored as graph nodes linking decisions, causes, outcomes, and precedents. This lets agents cite past logic, not just retrieve documents.
  • Graph embeddings enable structural + semantic similarity search: Text embeddings find similar words; graph embeddings vectorize entire decision traces (chains of reasoning), so agents retrieve past decisions that are both semantically relevant and structurally similar (same causal pattern). Hard to do with documents alone.
  • Live example—fraud detection agent: Queried customer data + decision traces, performed hybrid search on precedents (semantic + structural graph similarity), recommended rejecting a transaction with specific past cases as justification. Explainability baked in.
  • create-context-graph CLI tool ships full-stack agent apps in one command: Specify domain (healthcare, finserv, etc.), agent framework (Pydantic AI, LangGraph, Crew, etc.), get boilerplate backend, frontend, Neo4j graph, demo data, and reasoning scaffolding. 22 built-in domains, custom ontology generation, connectors for GitHub/Slack/Jira/Notion.
  • Neo4j agent memory package = short-term + long-term + reasoning memory API: Handles conversation history (short-term), entity extraction + deduplication across sessions (long-term), and decision trace storage (reasoning). Multi-stage NER pipeline (spaCy → GLiNER → LLM fallback) to pull entities from unstructured text. Integrates with Microsoft Agent Framework, Google ADK, etc.
  • This is v1 software: No automatic decision trace write-back yet (requires manual prompting), no temporal weighting on old vs. recent traces, custom domain setup still rough. Open source, actively evolving.

Useful details

  • Demo stack: Claude for agent runtime, OpenAI embeddings, Neo4j graph + vectors, Next.js frontend, fake CRM + support system data.
  • Cypher: Neo4j's graph query language (analogous to SQL for graphs).
  • Graph schema: Conversations (yellow nodes) → entities extracted → reasoning traces (decision nodes with causal edges: "caused," "next").
  • POLE ontology: Pre-defined entity/relationship types (Policy, Entity, etc.) used in the 22 built-in domains to guide extraction and mapping.
  • Graph Data Science (GDS): Neo4j library for generating graph embeddings (used in the fraud demo to find similar decision patterns).
  • CLI command example: uvx create-context-graph --name myapp --domain healthcare --framework pydantic-ai --demo-data → generates folder with JSON fixture data, graph schema, agent scaffolding.
  • Data connectors: Can import from GitHub, Notion, Jira, Slack instead of using demo data.

Caveats / counterpoints

  • Temporal relevance not yet implemented: No weighting for recent vs. old decision traces. Zach admits this is "something that will happen as the technology matures," but it's not there yet—potentially problematic for dynamic environments.
  • Decision trace creation is manual: Agents don't auto-store decisions unless prompted; write-back mechanism is "still being worked on." Limits autonomous learning loops.
  • Early-stage open source project: Zach emphasizes it's "still a little bit rougher on the edges," especially for custom domains and ontology generation from unstructured data. Expect bugs and missing features.
  • Structured vs. unstructured data gap: Extraction pipeline works well for text, but CSV/table mapping is a separate "just map Cypher statements" workflow. Not fully unified.
  • No discussion of cost, scale, or latency: Graph embeddings and hybrid search sound expensive; no benchmarks on query speed, storage overhead, or LLM token costs for multi-stage extraction.
  • Explainability claim undersold: Demo shows citing past decisions, but no evaluation of whether agents actually reason correctly or just pattern-match. "Explainability baked in" is a feature, not a proof.

Ken relevance

High relevance for agent systems and AI ops. Ken's building agents that need to make consistent, justifiable decisions (e.g., investment research, GTM workflows, content prioritization). Context graphs solve a real problem: agents currently have no memory of why they decided something, making multi-turn interactions brittle and opaque. The create-context-graph CLI tool is a fast prototyping wedge—Ken could spin up a domain-specific agent (e.g., "investing research") in an hour and see if decision tracing improves output quality or reduces hallucination. Graph embeddings for structural similarity are novel; if Ken's agents need to cite past reasoning (e.g., "I recommended this stock because of X precedent"), this is the stack.

Caution: This is v1, so don't bet the farm. But Neo4j's integration with LangGraph, Pydantic AI, and Microsoft Agent Framework means Ken can layer this into existing agent infra without rip-and-replace. The agent memory API (short/long/reasoning) is more mature than the CLI tool and could be useful standalone. Watch for temporal relevance and auto-trace-writing to ship; those are blockers for production use.

Investing angle: Neo4j is positioning graphs as the "knowledge layer" for agents. If decision tracing becomes table stakes for enterprise AI (explainability, compliance, audit trails), Neo4j is ahead. Keep an eye on adoption velocity of create-context-graph and whether competitors (Pinecone, Weaviate, LlamaIndex) add similar features.

Watch verdict

Skim. The core idea (decision traces as graph nodes, graph embeddings for structural similarity, hybrid search for precedents) is valuable and well-explained. The CLI tool demo is compelling for rapid prototyping. But there's repetition, the Q&A is low-signal, and Zach spends too much time on installation boilerplate. Watch the first 10 minutes for the concept and fraud demo, skim the rest unless you're implementing immediately.

Search this video Resume transcript at 0%
Full transcript 2784 words · 17 min read
0:00

Music

0:14

SPEAKER_00

So my name is Zach. I work for Neo4j. We're a graph intelligence company. You can think of us like a knowledge layer graph database at the core. We help connect and resolve information so AI systems can be a little bit more accurate and explainable. I used to work in technical marketing. I very recently am going back to my AI and machine learning roots and transferred over to a research engineering role. So I'm going to talk to you today about context graphs. How many people in this room have heard about context graphs before? Okay so we've got about half. And how many people have actually coded anything with a context graph or with a graph like Neo4j in general? Okay, okay about half. So let's go ahead and see what's going on. So I'm going to talk a little bit about how we think about context graphs at Neo4j, define what they are, and then I want to go over some tools that are built by William Loyne, who's one of our product managers, so that you can get started with them very quickly with your own code and your agent framework. So context graphs, foundation capital I think back in December and essentially we saw a lot of noise around this idea of creating decision traces and reasoning to help agents make better decisions. And so to think about what a context graph is, you need to ask ourselves what do agents really need to be accurate, right? And so for one thing, doing a lot of retrieval, you're going to need a knowledge base obviously. So a knowledge base for something like graph retrieval basically helps an agent or a chatbot answer questions correctly. What a context graph does in the evolution of this is really the information required to not only answer questions correctly but make better decisions. So if we took a concrete example and I'll show you a demo of this financial analyst agent and say that this agent has a question around improving increase and a request for a certain amount of money to allow that information about the system. So customer info, transactions, and policies. The response from that is likely to look something like this where it could maybe assign some sort of risk score and recommend some sort of review and talk about some key risk factors. What is helpful, what a context graph enables an agent to do is actually give an answer should you reject, accept, and why, right? And it does this because in addition to getting that customer information in those transactions, it's also going to get you past decision traces and precedents, dynamic information about why decisions are made. Systems of record really about facts, entities, state, about precedents, causal chains, expected outcomes, and enabling the agent to act with subject matter expertise and telling the agent really.

0:20

SPEAKER_00

Model for a context graph. The entities about things that exist. There'll be events, so decisions, transactions, approvals, things like that. And then context. And different reasoning by AI that records memory but by employees and past humans that have made decisions. And some of it because, you know, with internet connection, I never know how things are going to go. All the code for this is available. I'll show links at the end. Data that we generate, replicates, taking data in from a CRM and a support system in some other places. It's using Claude in the agent runtime. Has some Open AI embeddings. There's Neo4j database assisting the data with some vectors. And then we have our Next.js front end as well. So, let me show you. I just recorded. Running here so you can see what this looks like. Because again, these are going to go back. So you ask it a question. And it will call a series of tools. And you can see it bringing the graph data back. And then you see it getting these different decision traces. And eventually, it will get this reject decision. Right? And if that's a. Then it does this thing called find precedence. Which is going to be more in a second. But it's going to look at information in the graph to pull a bunch of.

0:25

SPEAKER_00

And in addition to the Internet would work, connect to each other in these causal chains. So you can build off of each other. So like I was saying before, we queried just context around Jessica's profile. We pulled back decision traces. And we did a special hybrid search, especially around those precedents, both on semantic similarity and structural similarity inside of the graph. That actually looked at how the decision traces were made from previous decisions. And it tried to match that. And I'll talk a little bit about how that works with graph embeddings in the next slide to ultimately come up with the recommendation. And so we have a vector index. So we're able to search. Fraud rejection, for example, on the semantics. But then we have this concept called graph embedding. So a lot of you in here are probably familiar with text embeddings, right, on words. A graph embedding is the same concept except those green nodes that I was showing you before everything was connected. We actually embedded those into a vector. And so what that means is similar decision traces are now going to be able to be looked up by vector similarity. And so if you have a system, right, where you've had past decisions and tickets that have been, now you can come up just like you would with vector search for another type of text. And that might be very hard to pull out if you just had it inside of documents. This is just some more information around the context graph demo and some of the scenarios you can run and some of the tools that were used here.

0:29

SPEAKER_00

[SPEAKER_02] GDS is called graph data science. [SPEAKER_02] That's what we use for the graph embeddings themselves.

0:39

SPEAKER_00

And then I'll link you over to the code here to rerun that. But what I really wanted to show you today was another tool that was just recently created a month ago, which allows you to create a full stack application to start with with just a one line command in the terminal to actually create the context ref and the front end and the back end. And so basically you can think about this like create React app or create Next app, like you basically get this boilerplate and all of the scaffolding that you use. And you can specify the domain when you create the graph or when you create the application rather and the command line. And it will give you this, basically the back end, the front end, and everything out of the box. So if I was to show you, if I go over here, it looks like this.

0:44

SPEAKER_00

A month ago, we released a tool that allows you to create a full stack application with just one line command in the terminal to create the context graph, the front end, and the back end. You can think about this like create React app or create Next app. You get this boilerplate and all of the scaffolding that you use. You can specify the domain when you create the application in the command line, and it will give you the back end, the front end, and everything out of the box.

0:59

SPEAKER_00

If I show you, it looks like this. I know it's probably hard to read. There's a UVX command: UVX create context graph. Specify the name of the app, the domain, the framework I want to use. There are a ton of them that we can use here. I'm choosing Pydantic AI and specifying demo data. It will create this folder for me with a bunch of fixture JSON data. In this case, dummy data that I can use to help host the application. We don't have enough time to run through all of them. It takes a few minutes to install the application, seed the data, and start. But I do have it running.

1:07

SPEAKER_00

It'll give you data inside of a graph, and then you can ask questions. It will do this type of retrieval with Cypher and everything else. Cypher is our graph query language to pull data back, very similar to what we saw in that context graph demo that was hosted. I'll let it run for a little bit here because we only have a few minutes. Can you hear me now? Okay. How long was that? Just a minute.

1:32

SPEAKER_00

All right. Well, here's our response back. You can see I asked it about prescription medications and it pulled everything back. There's a visualization of the schema here and different decision traces that you can look through. This is a great place to get started if you're interested in context graphs and want to code something up really quickly. There's a website for this too that explains step by step how to get the app started and explore everything. There are also some interesting new tools if you want to import from software as a service: GitHub, Notion, Jira, or Slack. You don't have to use just demo data. You can import data from these other tools.

1:42

SPEAKER_00

And there are other features. I'll give you a link to all of this at the end. We were looking at a Pydantic AI application, but you can also use OpenAI, LandGraph, Crew, Strands, Google ADK, and all these others. There are 22 built-in domains. Healthcare was the one I just showed, but there's also FinServe. You can also create your own custom domain, and it will help generate an ontology—basically a graph schema—for you and put everything together.

1:56

SPEAKER_00

We talked about the different data connectors for GitHub, Slack, and so on. It has all of the graph native functionality to power your queries to pull the data, look at decision traces, generate an MCP server, and do multi-turn conversation inside of that app. This project is just getting started, so it's open source. People are welcome to commit and contribute there. Now I'll talk about what this project is based on. One of the big dependencies is our Neo4j agent memory package, a complete memory API. Context graphs need all three of these things: short-term memory, long-term memory, and reasoning.

2:20

SPEAKER_00

Short-term is conversation history and session context. Long-term is the entities extracted from that and those that repeat over time and resolving those down. Reasoning is the traces inside of that context graph.

2:26

SPEAKER_00

Inside of here, which is very useful: a lot of people ask, if I have this text data, how do I put it into a knowledge graph? That's usually one of the big questions people have.

2:33

SPEAKER_00

We actually implemented this inside of the package, where it goes through a few different stages on raw text. There are other tools you can use outside of this if you want to take in structured data or use your own language model extraction or NER processes. But this one uses a few stages: spacey to gliner to more advanced to an LLM fallback. Then there's a separate merging, deduplication, and enrichment strategy. It's a little bit more well thought out, and that helps, especially with short-term memory, take that information out and transition it into useful long-term memory and entities that can be resolved over time.

2:37

SPEAKER_00

The schema looks like this: you have your conversations, and in yellow, you have your entities—your entities that get pulled out. Those connect to your reasoning traces. We're getting towards the end here, and we might have a few extra minutes for questions. Here are the resources. The context graph demo I showed: if you go to that blog, it'll also link you to a live application that's hosted. That's very useful if you want to explore the concepts out of the box. And then in yellow, you have your entities. Your entities that get pulled out.

3:12

SPEAKER_00

And those connect to your reasoning traces.

3:21

SPEAKER_00

Alright. So we're getting towards the end here.

3:33

SPEAKER_00

And we might actually have a few extra minutes for questions. If that's allowed. But here are the resources. So the context graph demo that I showed.

3:50

SPEAKER_00

If you go to that blog.

4:03

SPEAKER_00

It'll also link you to a live application. That's just hosted. And that's very useful.

4:18

SPEAKER_00

If you want to explore the concepts out of the box. Create context graph in the middle. On the top one. I know on our Wi-Fi. For this. For some reason it blocks that website. It won't block the website. If you go on a normal Wi-Fi network. I don't know why it does for this one. But that link does work. And then on the one on the bottom. Is just the GitHub repository. Neo4j agent memory. Is the agent memory package. That underpins all of that. And that also integrates with.

5:27

SPEAKER_00

Microsoft agent framework. As well as Google ADK. And a whole bunch of others.

5:52

SPEAKER_00

So that's actually it for me. I made it through with a couple minutes to spare. So I'm happy to open it up. To any questions for the next couple minutes. If there are any in the room. Yes. So you had to cause a lot of change. Yep. Is there any concept in there. And temporality. Like how long ago.

5:57

SPEAKER_02

[SPEAKER_00] These components happen.

5:57

SPEAKER_00

How relevant it is to. Retreat those at that time. Yeah. There is. You can add timestamps. You can add timestamps.

5:58

SPEAKER_02

[SPEAKER_00] And then.

5:58

SPEAKER_00

Obviously as they occur in time. The different steps. [SPEAKER_00] They'll be linked. [SPEAKER_00] By like caused or next. [SPEAKER_00] So it's a zip. [SPEAKER_00] But more weight.

6:13

[SPEAKER_00] Or more recent events. [SPEAKER_00] Or is that the following.

6:54

[SPEAKER_00] I don't know if it does that. [SPEAKER_00] Quite yet. [SPEAKER_00] I think that's something that will happen. [SPEAKER_00] As the technology matures a little bit. [SPEAKER_00] But yeah. [SPEAKER_00] That is a good point. [SPEAKER_00] It's a good idea. [SPEAKER_00] The ontology is fully general. [SPEAKER_00] It's just. [SPEAKER_00] You prepare the nodes. [SPEAKER_00] And the edges. [SPEAKER_00] The kinds of them. [SPEAKER_00] So you'll prepare the ontology beforehand. [SPEAKER_00] For those domains that. [SPEAKER_00] I showed you. [SPEAKER_00] The pre-existing ones.

8:07

SPEAKER_02

[SPEAKER_00] A lot of them will use. [SPEAKER_00] There's something called. [SPEAKER_00] POLE E. [SPEAKER_00] Which is. [SPEAKER_00] Entity.

8:29

SPEAKER_00

Policy. There's like. A few. Pre-defined. Entity. And relationship types. And that's used. To guide. The extraction. And all of the mapping. And stuff. So yes. Yes. If I have a. Giant amount of information. That's a little effort.

9:42

SPEAKER_00

Over time. Would I be able to. Somehow. Automate. Creating a front. Automate.

10:10

SPEAKER_00

Creating a front. Automate. Extracting.

10:25

SPEAKER_00

Information. Ensure. Turn it into. An ontology. If I'm telling you. Yeah. I would look at the. Create context graph package. At minimum. If you're able to describe the ontology. Then describe what's in the data. Then you can definitely create a graph schema from there. And then that will help you do. If it's unstructured. Do entity. Entity extraction. It depends on how structured your data is. So if it's very structured. And it's in. For example. CSV files. Or tables. Then it's just a matter of. Mapping. The cipher statements over. Yeah. [SPEAKER_00] So I think that. [SPEAKER_00] That tech stuff. [SPEAKER_00] I think that can be handled. [SPEAKER_00] Inside of here.

11:36

SPEAKER_00

[SPEAKER_00] If you look at. [SPEAKER_00] Create your custom domain. [SPEAKER_00] There might be some things that you can do there. [SPEAKER_00] To help you. [SPEAKER_00] Again. It's a new project. [SPEAKER_00] So. It's still a little bit rougher on the edges. But at least you'll see the code. And you'll see how. It can generate an example ontology. And then how that can be fed. To an extraction process. To move everything through. Yep. Over time. So there's. There's a way. To add.

12:51

SPEAKER_00

At the moment I know in this first demo that it is, you can ask it to store, like if I, for my previous conversation, I could ask it to store decisions, but I don't think it will do it unless I prompt it to. But yeah, that's a good point. And then in the create context graph, we're still working on how you would write new decision traces. So yeah, it's something to think about, some sort of sentiment or quality score on the decisions. All right. Cool. Well, thank you guys. People are welcome to commit and contribute there. Now, I'll talk a little bit about what this project is based on. So, underpinning this, one of the big dependencies is our Neo4j agent memory package.

13:15

SPEAKER_00

So, this is a complete memory API. It includes context graphs really need all three of these things. A short-term memory, a long-term memory, and reasoning. So, short-term is more conversation history and session context. I think we understand that. Long-term is the entities that are extracted from that. And those that repeat over time and resolving those down. And then the reasoning is going to really be sort of the traces inside of that context graph. And there's also inside of here, which is very useful. I know a lot of people think about, well, if I have this text data, how do I put it into a knowledge graph? Right?

13:56

SPEAKER_00

Is usually one of these big questions that people have. So, here we actually implemented this inside of the package. Where it goes through a few different stages on raw text. There's other tools that you can use sort of outside of this. If you want to take in structured data. Or you can use your own language model extraction, NER processes. But this one just uses a few stages where you go from spacey to gliner. And more advanced to an LLM fallback. And then there's a separate merging, deduplication, and enrichment strategy. So, it's a little bit more well thought out. And that helps, especially with the short-term memory.

14:35

SPEAKER_00

Take that information out and transition it into useful long-term memory. And entities that can be resolved over time. And the schema kind of looks like this. Where you have your conversations. And then in yellow, you have your entities. Your entities that get pulled out. And those connect to your reasoning traces. Alrighty. So, we're getting towards the end here. And we might actually have a few extra minutes for questions. If that's allowed. But here are the resources. So, the context graph demo that I showed. If you go to that blog. It'll also link you to a live application. That's just hosted. And so, that's very useful.

15:17

SPEAKER_00

If you just kind of want to explore the concepts out of the box. Create context graph in the middle. On the top one. I know on our Wi-Fi. For this. For some reason it blocks that website. It won't block the website. If you go on a normal Wi-Fi network. I don't know why it does for this one. But that link does work. And then on the one on the bottom. Is basically just the GitHub repository. Neo4j agent memory. Is the agent memory package. That underpins all of that. And that also integrates with. Microsoft agent framework. As well as Google ADK. And a whole bunch of others. So, that's actually it for me. I made it through with a couple minutes to spare.

16:02

SPEAKER_00

So, I'm happy to open it up. To any questions for the next couple minutes. If there are any in the room. Yes. So, you had to cause a lot of change. Yep. Is there any sort of concept in there. And temporality. Like how long ago. These components happen. How relevant it is to. Retreat those at that time. Yeah. There is. You can add timestamps. You can add timestamps. And then. Obviously as they occur in time. The different steps. They'll be linked. By like caused or next. So, it's a zip. But more weight. Or more recent events. Or is that the following. I don't know if it does that. Quite yet. I think that's something that will happen. As the. You know.

16:49

SPEAKER_00

As the technology matures a little bit. But yeah. That is a good point. It's a good idea. The ontology is fully general. It's just. You prepare the nodes. And the edges. The kinds of them. So, you'll prepare the ontology beforehand. For those domains that. I showed you. The pre-existing ones. A lot of them will use. There's something called. Pole E. Which is like. Entity. Basically. Policy. There's like. A few like. Pre-defined. Entity. And relationship types. And that's used. To guide. The extraction. And all of the mapping. And stuff. So, yes. Yes. If I have a. Giant amount of information. That's a little effort. Over time. Would I be able to. Somehow. Automate.

17:31

SPEAKER_00

Creating a front. Automate. Creating a front. Automate. Extracting. Information. Ensure. Turn it into. An ontology. If I'm telling you. Yeah. I would look at the. I would look at the. Create context graph package. At minimum. If you're able to describe the ontology. Then describe what's in the data. Then you can definitely create a graph schema from there. And then that will help you do. If it's unstructured. Do entity. Entity extraction. It depends on how structured your data is. So if it's like very structured. And it's in like. For example. Like CSV files. Or tables. Then it's just a matter of like. Mapping. The cipher statements over.

18:16

SPEAKER_00

Yeah. So I think that. That tech stuff. I think that can be handled. Inside of here. If you look at like. Create your custom domain. There might be some things that you can do there. To help you. Again. It's a new project. So. It's still a little bit rougher on the edges. But at least you'll see the code. And you'll see how. It can generate an example ontology. And then how that can be fed. To an extraction process. To move everything through. Yep. Kind of means like. Over time. So there's like. There's sort of a way. To add. To. To. To.

19:04

SPEAKER_00

To.

19:07

SPEAKER_00

To. To.

19:13

To.

19:16

To. To. at the moment. I know in this first demo that it is like you can ask it to store, like if I, like for my previous conversation, I could ask it to store decisions, but I don't think it will do it unless I prompt it to. But yeah, that's a good point. And then in the create context graph, we're still working on how you would write new decision traces. So yeah, it's something to think about, like some sort of sentiment or like quality score on the decisions.

19:52

All right. Cool. Well, thank you guys.

Reading tools

Type to find a passage

Appearance
Ask this transcript

Notebook

Export Markdown

Add a note