Connecting the Dots with Context Graphs — Stephen Chin, Neo4j

SPEAKER_00

Hello and welcome everybody to connecting the dots with context graphs. My name is Steven Chen. I run the developer relations team at Neo4j and you are in store for the power hour of context and graphs and all this technology. So I'm the first speaker. We have some other amazing talks after me, so I hope you enjoy all the great content which you're going to see over the next hour or so.

SPEAKER_00

So what I'm going to talk about is a bit about how we've all been feeling with the AI revolution, where we are trapped as engineers. We are using AI coding tools, or maybe they're using us, where our work is being reviewed. Who here has their work reviewed by an agent when they check in their PRs? Yes, all of you. So we are stuck in this limbo where we have amazing tools, we have amazing capabilities, but rather than us controlling them, they are controlling us. And we would like to get to a state where we're in control of this. So we have to decide: is it going to be the blue pill, where we're stuck inside of this mire of disparate knowledge, stuck in Slack discussions and little customer threads and different enterprise systems which are all segregated and siloed? And when we ask the agents to make critical business decisions or our applications to make critical business decisions with all this spread, it can't possibly give good answers because it doesn't have the context. Or do we want to dive in and embrace the red pill, escape from the matrix, and have a system of reasoning where we actually have all these systems connected, all of our different enterprise data sources, previous decision traces, the reasoning tool calls of the tools, to give us a more consolidated view of our enterprise stack and escape from the matrix? So who's in the escape club? Who wants to break out? Okay, hopefully if you're in the room, you're with me.

SPEAKER_00

And guess who else is with us? Gartner has now officially made context graphs as part of the AI hype cycle. So we have been officially recognized by the analysts of the world. They also realized that we're all stuck in this mire. Foundation Capital actually started this thread with their three trillion dollar startup opportunity post about how context graphs are going to move forward the industry and dramatically change how we build applications.

SPEAKER_00

And what I'll do is I'll show some demos and I'll talk about how we can move from being stuck in this matrix, stuck in this world, and then become the superheroes of our organization and actually build the capability, the systems, using technologies like knowledge graphs.

SPEAKER_00

So knowledge graphs are a very powerful tool for us to aggregate all this information, create the connections, create the relationships. And at a fundamental level, they hold nodes, which are people or things or companies, or relationships. You have relationships between nodes. In this case, you know, Dan, those are properties. Lives with. And he drives her car, apparently. So we know who wears the pants in this relationship. And we have some embeddings on top of the car. So we're embedding vector information in it. So we can also do similarity searches and combine the best of both worlds with building information but then also combining it with LLM.

SPEAKER_00

So when we take what LLMs are really good at—this language, this reasoning, this creativity—when we combine that with what knowledge graphs are really effective at—so knowledge, context, and enrichments—then we can start doing things with our data like storing all these relationships together, visualizing them so we can get to the data which matters, finding hidden patterns, and then analyzing this and getting more insights, which will help power the context graph demonstrations which I'm going to show you all.

SPEAKER_00

So here's a simple example of how graphs power retrieval because I think it's good to understand what the difference is between a baseline LLM. So this is a healthcare case. What was the care plan associated with Andreyi Jenkis's emphysema? And when you ask the LLM, it has broad knowledge. It understands a lot of information. It knows what emphysema is. It knows what standard practices are. So it gives a very generic answer: preventing damage to the lungs, and so on.

SPEAKER_00

Now when we give it a RAG system—so we go to a vector database—now it has more context. It knows a bit about the patient and their information. And it tells you maybe recommend some activities like respiratory therapy, deep breathing, coffee exercises. So this is pretty generic medical advice.

SPEAKER_00

Now, where we want to get to is grounded, complete information, where we're pulling in who's the patient, what was the previous diagnosis, what operations they have. And you can see here that it is specifically recommending medication management, smoking cessation counseling, pulmonary rehabilitation, exercise. So clearly, the patient here has a history of smoking, has had an operation. So there are certain things which are background information that was lost in the similarity search. And if our agents have this information, then like the matrix, now we've loaded all of this information, and we can, you know, we're like Neo, we can do kung-fu, we can shoot bullets, we can do all this amazing stuff with the right memory structure in place.

SPEAKER_00

So this is the second layer. So now we have the grounding with graph RAG and retrieval, and we can pull things out of knowledge graphs. But we need to now store the memories—the short-term memory, the long-term memory, and the reasoning memory—so that we have our complete history of what's happened, that we can build on top of this. So I think you all know that short-term memory is things which are happening in the current pipeline with agents, the conversation, the current state of activities which your agent architecture is working on. So this can all be persisted in the knowledge graph and it gives important information in the execution pipeline.

SPEAKER_00

Long-term memory is really important and needs to be organized well because there's so much of it. So you have to figure out how to aggregate and pull this information and have a good domain model for representing the different business processes, the entities, the folks who are part of your application or part of your domain. And then you can actually store the information which your agents are working on over longer tasks and procedures and across multiple different user or customer interactions, to give that history and context for what your application has done.

SPEAKER_00

And then finally, reasoning traces give us the ability to understand why decisions were made and how they're done. So typically what we get from LLMs is we get the result right. They'll tell us: well, this is what I recommend, this is what I advise. But to get to that result, there is thinking, there is reasoning which happens behind the scenes. And we'd like to make it repeatable, where that information is learning from the experience of the previous traces. It has that decision provenance. So if there were previous decisions, you're going to draw on that knowledge and we'll use that to come up with better future decisions. And this also gives us a great hook-in for compliance and debugging.

SPEAKER_00

One of the great things about knowledge graphs is they're great for tooling. They're great for LLMs. LLMs can build Cypher, which is the query language for knowledge graphs. They can create knowledge graphs and create structure out of unstructured documents.

SPEAKER_00

To get to that result there is thinking, there is reasoning which happens behind the scenes, and we'd like to make it repeatable where that information is learning from the experience of the previous traces it has. That decision provenance, if there were previous decisions you're going to draw on that knowledge, and we'll use that to come up with better future decisions. And this also gives us a great hook-in for compliance and debugging. One of the great things about knowledge graphs is they're great for tooling. They're great for LLMs. LLMs can build Cypher, which is the query language for knowledge graphs. They can create knowledge graphs and create structure out of unstructured documents. But it's also how we would represent things as humans. It's how we would draw things on a whiteboard. That's how we would show things. And with these, now we are loading up the memory into a structure where we can actually start to do interesting things with our memory. Graphs are a great use case for memory because relationships are first-class within knowledge graphs. They're part of the structure. It's not like you have to join a bunch of tables together. It's a natural progression. They're highly performant for multi-hop traversal. A lot of the graph research papers talk about this specifically as a major advantage to graph research and graph AI architectures where they can navigate more complex structures at a very high performance speed. Using graph embeddings like FastRP, we can also do vector lookups, which is a great way to get a starting point or hook into the graph where we navigate using algorithms like the Louvain algorithm for community grouping. And then we get explainable decisions. We have more cross-domain knowledge, and we're building ACID-compliant solutions with things like the Neo4j agent memory package. This is an open source package which we built on top of Neo4j. We have an open GitHub repo. We encourage other folks to contribute to it. And it brings these three concepts together: short-term memory, long-term memory, and reasoning into a context graph structure. The first demonstration I'm going to show is an example of how you can build a knowledge graph which ties your short-term, your long-term memory, and your reasoning memory together to answer questions from Lenny's podcast. Who's a fan of Lenny's podcast in the room? A bunch of folks. It's a great resource. But it's hard. Podcasts are hard. They're very dense. There's a lot of connected information and topics, and we'd like to be able to extract that and then understand more of the context and things which are happening with the help of AI. So we built a little demo: Lenny Memory Podcast, again an open source project. What it lets you do is it has all the podcasts loaded up. For those of you who don't know, Lenny talks a lot about different AI topics and about product management. And one of the things we can do is we provide the AI with different tools for accessing the memory. This is all written on top of the Neo4j agent memory APIs. And then, for example, we're pulling back locations in the episode, and it's using that to design and build a graph or in this case a map and show us all the different locations of things mentioned in the film by aggregating all that context. And because we have it in a graph format, it's not just pulling out some similar locations and getting part of the data. We get a holistic view of the entire data set which can be navigated and queried dynamically. So now we've shown what we can do with graphs, what we can do with memory. But what we're all here for is context graphs, right? How can we take this and actually apply this to solve those cross-domain business problems where it's very hard to get the information, it's very hard to quantify why decisions are made? Context graphs are really powerful for this because, unlike a traditional audit log, they're capturing the why. The decision traces that happen while you're evaluating your models. It organizes these by entities and relationships, and then it's pulling all the knowledge from different sources. So rather than having conversations hidden in Slack or emails or other informal conversations, now your app becomes a central point where they can look up previous decisions. They can get that advice, and then they can add that recommendation back to the reasoning traces for future lookups. Broadly, the architecture is: you're searching using your context graph retrieval tools from your agentic architecture. It's using a combination of knowledge graphs, vector search, and data science algorithms. Then, when you go through the agent loop, it's pushing that back into the context memory, which gets added back into the graph. And subsequent queries are then pulling this back as part of your reasoning traces and your output to solve specific domain problems. So what I'm going to show here is an example from a financial services application. We're going to have entities of different people and organizations, different events for decisions, transactions, and approvals which happen during the workflow of the application, and then the context of why, what policies were applied, what risk factors are there, what was the employee reasoning behind giving a certain recommendation. And the architecture, again, it's an open source project you can try this out on. We have a hosted version of this, and you can try it out with the GitHub project and run locally. But it's pulling in from a variety of different data sources. So we've hooked it up to a support ticket system, a CRM, and an internal business data system with ten different MCP tools that it has access to. And then we've used Claude agent to create OpenAI embeddings and then populate a Neo4j context graph with a lot of this information so that it has a domain graph and a reasoning graph which it can look into. And then finally, it's exposed with a user interface which is a simple Next.js application that gives us a front end like what we want an end user or consumer to use for this particular use case. And for this application, what it does is it presents to you a prompt where you can ask it a bunch of questions. We're going to ask it about Jessica Norris and see whether she should get an approval. And it's going back to the graph, and it's querying both information about her history. So it knows what her bank account is. It knows that she has some related margin trades. You can see some of the Cypher queries there that were queried through the model. And you can also see the knowledge graph that we're traversing and populating. So this is why knowledge graphs make things explainable and auditable because now we see exactly the information which is being populated and used. You can see there's a previous rejection, and these are the sort of things which get lost in disparate systems and tooling where we don't bring all this information together in a queryable, understandable form where we can build and pull out that knowledge. Now, unfortunately for Jessica, the AI model recommends not giving her the loan. But it gives us the reasons. The risk factors. It gives us previous decisions which should influence this and fraud detection patterns about why this may be a big risk for our organization. And as an agent, or as a user, a human who's using the system to make decisions, this is the sort of information you need to actually make a decision you can stand behind and you can justify to your organization. And then us as developers, now we can justify why our agentic applications are actually solving real business problems, providing grounded information that our users can rely on and are taking advantage of the latest techniques with context graphs, which, as we know, Gartner approves of. Alright, so let me leave you with some resources that you can use to learn more. I run the DevRel team. One of our big pushes is free education. So we just want to help people understand how to use graphs, how to use context graphs.

SPEAKER_00

patterns about why this may be a big risk for our organization and as an agent or a user, a human who's using the system to make decisions, this is the sort of information you need to actually make a decision you can stand behind and you can justify to your organization. And then us as developers, we can justify why our agentic applications are actually solving real business problems. Providing grounded information that our users can rely on and are taking advantage of the latest techniques with context graphs, which as we know Gartner approves of.

SPEAKER_00

Alright, so let me leave you with some resources that you can use to learn more. I run the DevRel team. One of our big pushes is free education, so we just want to help people understand how to use graphs, how to use context graphs, and how to use AI. We have a new context graph course that we just released on Graph Academy. Also, it makes it really easy to get started because we spin up a free Aura instance in the background, so you have a graph database to play with for free. You can try a bunch of these techniques out before you even try it in your own production instance or your own enterprise instance.

SPEAKER_00

So I hope you guys enjoyed the talk and learned a little bit about the possibilities with context graphs. The next set of presenters, my colleagues Zaid and ABK, are going to dig a little bit more into agentic use cases of context graphs. And please come chat with us either after the talk or at the Neo4j booth. We're happy to have conversations, dig more into demos, dig more into your use cases, and help all of us escape the matrix. So thank you very much. our different enterprise data sources previous decision traces the reasoning tool calls of the tools to give us a more consolidated view of our enterprise

SPEAKER_00

stack and escape from the matrix so who's who's gonna who's in the escape club who wants to break out okay hopefully if you're in the room you're you're with me and guess who else is with us Gartner has now officially made context graphs as part of the AI hype cycle so we have been officially recognized by the the analysts of the world they also realized that we're all stuck in this in this mire foundation capital actually started this thread with their three trillion dollar startup opportunity post about how context graphs are going to move forward the industry and dramatically change how we build applications and what I'll do is I'll show some demos and

SPEAKER_00

I'll talk about how we can move from being stuck in this matrix stuck in this world and then become the superheroes of our organization and actually build the capability capability the systems using technologies like knowledge graphs so knowledge graphs are a very powerful tool for us to aggregate all this information create the connections create the relationships and at a fundamental level they they hold nodes which are people or things or companies or relationships you have relationships between nodes where in this case you know Dan those are properties lives with and they he drives her car apparently so we know who who wears the pants in this relationship and we have

SPEAKER_00

some embeddings on top of the car so we're embedding vector information in it so we can also do similarity searches and kind of combine the best of both worlds with building information but then also combining it with LLM so when we take what LLM's are really good at this language this reasoning this creativity when we combine that with what knowledge graphs are really effective at so knowledge context and enrichments then we can start doing things with our data like storing all these relationships together visualizing them so we can get to the data which matters finding hidden patterns and then analyzing this and getting more insights

SPEAKER_00

which will help power the context graph demonstrations which I'm going to show you all so here's a simple example of how graphs power retrieval because I think it's good to understand what the difference is between a baseline LLM so this is a healthcare case what was the care plan associated with Andreyi Jenkis emphysema and when you ask the LLM it has broad knowledge it understands a lot of information it knows what emphysema is it knows what standard practices so it gives a a very generic answer preventing damage to the lungs yada yada yada now when we give it a rag system so we go to vector database now it has more

SPEAKER_00

context it knows a bit about the the patient and their information and it tells you maybe recommend some activities like respiratory theory deep breathing coffee exercises so this is pretty generic medical advice now where we want to get to is grounded complete information where we're pulling in who's the patient what was the previous diagnosis what operations how they have and you can see here that it is specifically recommending medication management smoking cessation counseling pulmonary rehabilitation exercise so that clearly the parent here has a the patient here has a history of of smoking has had an

SPEAKER_00

operation so like there's there is certain things which are background information that was lost in the similarity search and if our agents have this information then like the matrix now we've loaded all about this information and we can you know we're like Neo we can do kung-fu we can shoot bullets we can do all this amazing stuff with the right memory structure in place so this is kind of the second layer so now we have the the grounding with with graph rag and retrieval and we can pull things out of knowledge graphs but we need to now store the the memories the short-term memory the long-term memory and the reasoning

SPEAKER_00

memory so that we have our complete history of what's happened that we can build on top of this so I think you all know that short-term memory is things which are happening in the current pipeline with agents the conversation the the current state of activities which are your agent architecture is working on so this can all be persisted in the knowledge graph and it gives important information in the execution pipeline long-term memory is really important and needs to be organized well because there's so much of it so you have to figure out how to aggregate and pull this information and have a good domain model for

SPEAKER_00

representing the the different business processes the entities the folks who are part of your application or part of your domain and then you can actually store the information which your agents are working on over longer tasks and procedures and across multiple different user or customer interactions to give that history and context for what your application has done and then finally reasoning traces give us the ability to understand why decisions were made and how they're done so typically what we get from LLMs is we we get the result right they'll tell us well this is what I recommend this is you know advise this but

SPEAKER_00

to get to that result there is there's thinking there's reasoning which happens behind the scenes and we'd like to make it repeatable where that information is learning from the experience of the previous traces it has that decision provenance for if there were previous decisions you're going to draw on that knowledge and we'll use that to come up with better future decisions and this also gives us a great hook-in for compliance and debugging one of the great things about knowledge graphs is they're great for tooling they're great for LLMs LLMs can build cipher which is the query language for knowledge graphs they can

SPEAKER_00

create knowledge graphs and create structure out of unstructured documents but it's also how we would represent things as humans it's how we would draw things on a whiteboard that's how we would show things and with these now we are loading up the memory into a structure where we can actually start to do interesting things with our memory so graphs are a great use case for memory because relationships are first-class within knowledge graphs they're part of the structure it's not like you have to join a bunch of tables together it's a natural progression they're highly performant for multi-hop traversal a lot of the graph reg research

SPEAKER_00

papers talk about this specifically as a major advantage to graph reg and graph AI architectures where they can navigate more complex structures at a very performance speed using graph embeddings like fast RP we can also do vector lookups which is a great way to get a starting point or hook into the graph where we navigate using algorithms like the low vein algorithm for community grouping and then we get explainable decisions we have more cross knowledge and we're building acid compliant solutions with things like the neo4j agent memory package so this is an open source package which we built on top of neo4j we have an open

SPEAKER_00

github repo we encourage other folks to contribute for it and it brings these three concepts together short-term memory long-term memory reasoning into a context graph structure and the first demonstration I'm going to show is an example of how you can build a knowledge graph which ties your short-term your long-term memory and your reasoning memory together to answer questions from Lenny's podcast who's who's a fan of Lenny's podcast in the room okay a bunch of folks it's a great resource and but it's hard like podcasts are hard they're very dense there's a lot of connected information and topics and we'd like to

SPEAKER_00

be able to extract that and then understand more of the context and things which are happening with the help of AI so we built a little demo Lenny memory podcast again an open source project what it lets you do is it has all the podcasts loaded up for those of you don't know Lenny talks a lot about different AI topics about product management and one of the things we can do is we provide the AI with different tools for accessing the memory this is all written on top of the neo4j agent memory API's and then for example we're pulling back locations in the episode and it's using that to design and build a graph and or in this case a map

SPEAKER_00

and show us all the different locations of things mentioned in the film by aggregating all that context and because we have it in a graph format it's not just pulling out some similar locations and getting like part of the data we get a holistic view of the entire data set which can be navigated and queried dynamically so now we've shown what we can do with graphs what we can do with memory but what we're all here for is is context graphs right how can we take this and actually apply this to solve those cross domain business problems where it's very hard to get the information it's very hard to

SPEAKER_00

quantify why decisions are made and context graphs are really powerful for this because unlike a traditional audit log they're capturing the the why the decision traces that happens while you're evaluating your models it organizes these by entities and relationships and then it's pulling all the knowledge from from different sources so rather than having conversations hidden in slack or emails or other informal conversations now your app becomes a central point where they can look up previous decisions they can get that advice and then they can add that recommendation back to the reasoning traces for future lookups

SPEAKER_00

broadly the architecture is you're you're searching you're using your context graph retrieval tools from your agentic architecture it's using a combination of knowledge graphs vector search and data science algorithms then the when you go through the agent loop it's then pushing that back into the context memory which gets added back into the graph and subsequent queries are then pulling this back as part of your reasoning traces and your output to solve specific demand problems so what I'm going to show here is an example is a financial services application for this we're going to have entities of

SPEAKER_00

different people and organizations different events for decisions transactions and approvals which happen during the workflow of the application and then the context of why what policies were applied what risk factors are there what was the employee reasoning behind giving a certain recommendation and the architecture again it's an open source project you can try this out on we have a hosted version of this and you can try it out with the github project and run locally but it's pulling in from a variety of different data sources so we've hooked it up to a support ticket system a CRM and an internal

SPEAKER_00

business data system with 10 different MCP tools that it has access to and then we've used cloud agent to create open AI embeddings and then populate a Neo4j context graph with a lot of this information so that it has a domain graph and a reasoning graph which it can look into and then finally it's exposed with a user interface which is a simple next JS application that gives us a front end like what we want an end user or consumer to use for this particular use case and for this application what it does is it presents to you a prompt where you can ask it a bunch of questions we're going to ask it about Jessica Norris and see whether she should get an approval

SPEAKER_00

and it's going back to the graph and it's querying both information about her history so it it knows what her bank account is it knows that she has some related margin trades you can see some of the cipher queries there that were queried through the model and you can also see the knowledge graph that we're traversing and populating so this is why knowledge graphs make things explainable and auditable because now we see exactly the information which is being populated and used you can see there's a previous rejection

SPEAKER_00

and these are the sort of things which get lost in disparate systems and tooling where we don't bring all this information together in a queriable in a understandable form where we can build and pull out that knowledge now unfortunately for Jessica the AO model recommends not giving her the loan but it gives us the reasons the risk factors it gives us previous decisions which should influence this and fraud detection

SPEAKER_00

patterns about why this may be a big risk for our organization and as a an agent who's or a user a human who's using the system to make decisions this is the sort of information you need to actually make a decision you can stand behind and you can justify to your organization and then us as developers now we can justify why our agentic applications are actually solving real business problems providing grounded information that our users can rely on and are taking advantage of the latest techniques with context graphs which as we know Gartner approves of

SPEAKER_00

alright so let me leave you with some resources that you can use to learn more I run the DevRel team one of our big pushes is free education so we just want to help people to understand how to how to use graphs how to use context graphs how to use context graphs how to use AI we have a new context graph course that we just released on graph academy also it makes it really easy to get started because we in the background we spin up a free aura instance so you just have a graph database to play with for free you can try a bunch of these techniques out before you even try it in you know your own production instance your own enterprise instance

SPEAKER_00

So I hope you guys enjoyed the talk and learned a little bit about what the possibility with context graphs is the next set of presenters my my colleagues Zaid and ABK are going to dig a little bit more into agentic use cases of context graphs and you know please come chat with us either after the talk or at the Neo4j booth we're we're happy to have conversations kind of dig more into demos dig more into your use cases and help all of us to escape the matrix so thank you very much