Thinner Agents on a Smarter Substrate: The Ontology-based Semantic Layer — Emil Eifrem, Neo4j
Description
To automate opening a bank account, your agent needs to verify identity, so a team wires it to the DMV and a passport service and ships it. Then the next team builds the next agent and rediscovers, from scratch, where its data lives, across a hundred databases plus Snowflake, Databricks, and S3, whether it can trust the version, and whether it is even allowed to touch it. Every agent repeats that wiring, nothing updates when a source moves without a manual rewire, and no agent is smarter tomorrow than today. Emil Eifrem's fix is to make the agents thin and put the intelligence in a shared substrate underneath. That substrate is an ontology based semantic layer with three parts. A business ontology names the real concepts, customers, accounts, checks, in the words people actually use, not f_name. A technical ontology catalogs every data source and its schema, with a mapping between the two. And execution traces record what each agent tried and whether it worked, so the layer learns bottom up: an agent that succeeded with the DMV lookup last time is more likely to reach for it next time. Discovery, trust, deduplication, and learning stop being every team's problem and become the substrate's. Speaker info: - https://x.com/emileifrem - https://www.linkedin.com/in/emileifrem/ Timestamps: 0:00 - The account opening agent and its data sources 1:53 - The problem: every team rewires data from scratch 4:00 - Thin agents on a smarter shared substrate 4:37 - Pillar 1: a business facing ontology 5:26 - Pillar 2: a technical ontology and the mapping 6:19 - Pillar 3: execution traces that make it learn 8:01 - Solving discovery, trust, DRY, and learning
Summary
Generated by gpt-5.6-terraAt-a-Glance
- Verdict: Skim
- Core thesis: Enterprise agent systems should shift from individually data-wired “thick agents” to thin agents that operate over a shared, governed ontology-based semantic layer linking business concepts, technical data assets, and execution outcomes.
- Why it matters: The architecture addresses a central scaling problem for agent operations: discovering trustworthy, authorized data and updating agent behavior across a fragmented enterprise without reimplementing integrations and knowledge in every agent.
- Best use: Use this as a concise architectural framing for a shared agent-data control plane; skim for the three-layer model rather than expecting implementation-level guidance.
Executive Summary
Eifrem argues that the limiting factor for enterprise agents is not only reasoning or orchestration logic, but the repeated work of connecting each agent to the right data. A simple account-opening agent may need identity validation from DMV and passport systems, but a large enterprise contains hundreds of databases, warehouses, object stores, duplicated records, differing schemas, permissions, and contested systems of record. If each team embeds that discovery and integration logic in prompts and code, every agent becomes a brittle, independently maintained integration project.
Neo4j’s proposed answer is “thin agents on a smarter shared substrate”: retain lightweight agent logic while centralizing the relationship between business intent and operational data in an ontology-based semantic layer. The business ontology names concepts in business terms—such as customer, account, transaction, and compliance process—while the technical ontology records the actual systems, schemas, and locations behind those concepts. A mapping connects the two, such as mapping a customer’s business-level first name to an Oracle field named F_name.
The third component is runtime execution traces. Agents record what action they attempted, the context, whether it succeeded, and the outcome; these traces can yield scores that guide future source selection. In the example, repeated successful DMV identity lookups make that source more likely to be selected under comparable future conditions. This turns successful experience into shared, cross-agent operational knowledge rather than leaving it trapped in a single agent’s code or prompt.
The talk is strategically useful but intentionally high-level and vendor-framed. It articulates a credible control-plane pattern for multi-agent, multi-source organizations, including governance and learning loops, but does not explain the stated three methods for constructing the technical ontology, scoring methodology, access-control enforcement, evaluation design, or how to resolve conflicts between freshness, reliability, cost, and compliance.
Key Takeaways
- Claim: Manually wiring each agent to enterprise data sources creates a scaling and maintenance failure, even when individual agents work initially. | Evidence: Eifrem contrasts a startup with one application and one Postgres database against an enterprise with “a hundred databases,” Snowflake, Databricks, S3, and duplicated data. Each agent team must rediscover where data lives, whether it is the correct version, whether it is trusted, and whether access is allowed. | Implication: Treat data discovery, provenance, authorization, and system-of-record selection as reusable platform capabilities—not per-agent prompt engineering tasks. | Caveat: The speaker explicitly limits this as primarily an enterprise problem; a small startup with a simple, unified data environment may not need this level of shared semantic infrastructure.
- Claim: A shared semantic layer should have a business-facing ontology, a technical ontology, and runtime execution traces. | Evidence: The business ontology models concepts such as customers, accounts, debit cards, checks, and transactions in terms humans recognize. The technical ontology captures metadata across assets—Oracle and Neo4j databases, Snowflake, Databricks, and S3—including where they are and their schemas. Runtime traces record what an agent tried, its context, success, and outcome. | Implication: A viable agent substrate needs both a human-governable semantic model and machine-operable metadata, augmented by observed execution performance. | Caveat: The talk says the technical ontology can be constructed in three key ways but omits those methods.
- Claim: Mapping business concepts to their technical systems of record removes repeated data-integration logic from agents. | Evidence: The speaker’s example maps a business-level customer first name to a specific Oracle database column, F_name. For compliance, the graph maps the need to resolve a government-issued ID to two available sources: motor vehicle records and passport verification. | Implication: Agents can invoke capabilities based on semantic intent—such as “resolve government-issued ID”—rather than carrying hard-coded knowledge of schemas, databases, and endpoints.
- Claim: Execution traces can turn agent behavior into a shared learning loop for selecting data sources or actions. | Evidence: For each graph traversal and execution, the system records where the agent was, what it did, the context, and whether it succeeded. Eifrem says these signals ultimately produce a score; repeated success with DMV lookup makes that option more likely to be chosen in the proper future context. | Implication: Instrumenting tool and data-source calls is not merely observability; it can support controlled routing policies that improve across agents—provided learning remains constrained by governance. | Caveat: The presentation does not specify how success is measured, how scores account for failure modes or policy constraints, or whether learned preferences are reviewed before deployment.
- Claim: The semantic layer provides a governed change surface that can propagate data changes across many agents while preserving trust signals. | Evidence: Eifrem claims the three-pillar model solves discovery, trust, duplicated wiring, and lack of learning: humans can curate trust top-down, execution traces provide bottom-up evidence of what worked, and a single mapping layer lets changes cascade rather than requiring manual rewiring of every agent. | Implication: Centralize semantic mappings and policy metadata, but manage the layer itself like production infrastructure with ownership, auditability, tests, and safe rollout controls. | Caveat: A single shared layer also becomes a critical governance and reliability dependency; the talk does not cover versioning, approval workflows, rollback, or blast-radius management.
- Claim: Markdown-based skills and prompt documentation are useful but insufficient as the sole mechanism for enterprise data access and agent knowledge. | Evidence: Eifrem says teams have attempted to solve the issue with markdown files alone, but concludes that they are “part of the solution” rather than the solution; he cites a recent Latent Space podcast comment that teams must learn their databases and cannot “vibe code with just markdown files.” | Implication: Keep human-readable skills and documentation, but back them with structured, queryable metadata and runtime evidence if agents must operate reliably across changing systems. | Caveat: The argument is asserted rather than demonstrated through a comparative implementation or quantitative evidence.
Detailed Brief
Account-opening example: process-guided traversal rather than free-form tool selection
- Claims: The proposed graph can encode not just entities and data mappings but also an explicit business process for agents expected to follow prescribed workflows.; For a compliance step in account opening, the agent identifies a required concept—government-issued ID—then discovers suitable technical sources through the semantic mapping.
- Evidence: The example combines business nodes including checks, accounts, and credit history, and calls the account-opening system a “process-following” or “process-guided” agent.; The compliance check node links to motor vehicle records and passport verification as alternative ways to resolve an ID.
- Caveats: The transcript does not distinguish which process decisions are deterministic policy requirements versus choices delegated to the model.; It does not address exception handling when both sources disagree, are unavailable, or yield uncertain results.
- Implications: The pattern is particularly relevant for regulated workflows where agents should navigate an approved process model but still need contextual flexibility in choosing data sources and tools.
Evidence level and commercial framing
- Claims: Neo4j presents the model as an emerging pattern observed with a Fortune 20 global bank, a Bay Area technology platform, and a leading fintech company.; The talk positions graph-plus-AI patterns broadly, but offers this semantic-layer architecture as its principal concrete pattern.
- Evidence: The named organizations are not disclosed, and no deployment metrics, architecture diagrams beyond the simplified conceptual graph, or operational results are presented.; The closing directs viewers to Neo4j documentation, its Expo booth, a graph-track conference program, and a startup program with credits and solution-engineering support.
- Caveats: Claims of traction and enterprise use should be treated as vendor assertions without enough detail to assess adoption scope, ROI, or the necessity of Neo4j specifically.
- Implications: Separate the reusable architectural principle—a shared semantic and execution substrate—from the vendor selection question; compare graph, catalog, knowledge-graph, and policy-layer approaches against concrete workload requirements.
Notable Concepts & Terms
- Thin agents on a smarter shared substrate: The central architecture: simplify individual agents by externalizing data knowledge, mappings, governance, and accumulated execution experience into a shared platform layer.
- Business-facing ontology: A human-legible model of organizational concepts, actions, and relationships, expressed as customer and first name rather than database-specific fields such as F_name.
- Technical ontology: A metadata model of enterprise data assets, including locations, schemas, and systems such as Oracle, Neo4j, Snowflake, Databricks, and S3.
- Business-to-technical mapping: The governed link between a semantic business need or entity attribute and its underlying systems of record, interfaces, or fields.
- Execution traces: Runtime records of agent actions, context, success, and outcomes that can inform future routing and become shared cross-agent learning.
- Process-guided agent: An agent intended to follow an encoded business workflow, such as account-opening compliance, rather than freely choosing its entire sequence of actions.
- DRY principle: “Don’t repeat yourself”; Eifrem uses it to argue against duplicating source-selection and integration logic across agents.
Operator Notes / Why Ken Should Care
- Define a minimal semantic contract for shared agent tools: business capability, required inputs, candidate systems of record, data classification, access policy, owner, freshness expectation, and fallback path.
- Instrument every agent-to-tool and agent-to-data call with structured context, chosen source, result quality, latency, failure class, and human override; do not rely on unstructured traces if the goal is reusable routing intelligence.
- Keep routing learned from outcomes bounded by hard policy gates for authorization, compliance, and source eligibility; never let historical success override permission or regulatory constraints.
- Run a design comparison before adopting a graph database: assess whether an existing catalog, service registry, policy engine, metadata store, or knowledge graph can implement the required mappings and governance with lower operational cost.
- Require versioning, ownership, tests, and rollback for semantic mappings, because a shared mapping update can alter behavior across every dependent agent.
Source/Metadata
- Title: Thinner Agents on a Smarter Substrate: The Ontology-based Semantic Layer — Emil Eifrem, Neo4j
- Transcript words: 3387
- Duration seconds: 666
- Timestamp note: No timestamps or chapters were present. The transcript contains a near-verbatim repeated second half, likely from extraction duplication.
Transcript
. All right. At Neo4j, we work with some of the largest companies in the world to help make their data ready for AI agents. And today I want to talk to you about a problem that we saw emerging over the last, call it, six to nine months, and propose a solution blueprint for that. So let's say that we work at a bigger organization, a big bank, and we want to write an agent. Let's say that agent is helping automate the opening of a bank account. Right? You can imagine that's very ripe for automation. You want to be able to orchestrate that process. And I'm going to use the powers bestowed upon me by a short keynote slot to grossly simplify what that agent looks like. I'm going to say there are two pieces. The first one is, let's call it the business logic. Some version of interpreting intent and plan, act, and we loop around that. It's what your agent does. And we know that when an agent acts, it doesn't always operate on data, but we equally know that in order for agents to be successful, a huge part of that is giving it access to the right data at the right time. So the second big bucket is, let's call it the data sources. You need to identify, figure out, okay, in order to solve my problem, I need access to these few things, and wire them up and make them available to the agent. In the example of our account opening agent, maybe we can imagine that we need to be able to validate identity. And so we might look at two data sources for that: the Department of Motor Vehicles, the DMV registry, and maybe some kind of passport verification service. So we wire that up into our agent, and it works. It's great. It's fantastic. And at the same time, you and other teams in your organization are building other agents, and conceptually they look very similar. So that's great. It's fantastic. It works. But it has a few problems. So first of all, every single time a team has to build an agent, they have to figure out from scratch where the data that they require for that agent to operate sits, which, if you work at a startup and you have one application that sits on top of one Postgres database, that's not hard. The data is in that Postgres database. But in an enterprise ecosystem, you don't have one database, you have a hundred databases, and you have Snowflake and Databricks, probably, and you have S3 buckets, and so on and so forth. You have to do that work manually from scratch every single time. And then, when you've found the data sources, in an enterprise, there's lots of duplication of data, so they need to figure out, is this the right data? Is it the right version? Can I trust it? Am I allowed to access it? So on and so forth. It also violates one of the core principles of software engineering, the DRY principle: don't repeat yourself. So when something changes, that cascades across all of your agents. You have to manually rewire all of them all the time, which works, but it's just a lot of work. And then finally, there's no learning around the data sources and how your agents operate on them. So when your agent wakes up tomorrow, it's not smarter than it was today, and there certainly isn't any cross-agent learning, because all of that wiring between business intent and the data sources is encoded in a combination of code and prompts. So I know what you're all thinking. Markdown files, skills to the rescue. And yes and no. You can come talk to me afterwards for the full version of this, but we've seen a ton of teams that tried to solve this problem using just markdown files. And the summary is, it is part of the solution, but it is not the solution. But don't take it from me, take it from Surov. A week ago on the Latent Space podcast, I said, hey, guys, you've got to learn your databases. You cannot vibe code with just markdown files. So we've been solving this problem at scale for some really massive organizations recently, including a Fortune 20 global bank, a massive tech platform company based here in the Bay Area, and a leading fintech company. And the pattern that is emerging is that in order to do agents at scale, we need thin agents on a smarter shared substrate. Thin agents on a smarter shared substrate. And what does that look like in practice? There are three pillars to that. The first pillar is a business-facing ontology. And the word ontology, I grew up in this world, people talked about ontology forever. More recently, it's become very hype, probably thanks to Palantir, but also the rise of AI. And there are a lot of people who want to make ontology really complex. But the core concepts are actually super simple. What are the key concepts in your organization? What are the key things that you can do to make a business organization? In our banking example: customers, accounts, debit cards, checks, transactions, and how do they all relate? But very importantly, they are expressed in a way that makes sense to all the human beings working in your universe, right? All the people working in your company, it's expressed in that way. In other words, you don't say F underscore name. No, you have a customer and they have a first name. So that's the first: a business-facing ontology. The second pillar is a technical ontology. This is all the metadata of all the data sources and data assets in your enterprise ecosystem. I have 14 Oracle databases, I have 15 Neo4j databases, I have Snowflake and Databricks, and I have S3 buckets, and all of that kind of stuff. Where do they sit? What are the schemas? All of that kind of good stuff. You construct that technical ontology in three key ways that we can talk about later, though not in this talk. And then you have a mapping between the two. So that customer that has a first name, that first name has a system of record, and over there, there's an Oracle database with a column called F underscore name, the mapping between the two. And then the third pillar is the runtime signals out of your agents. When they walk this graph and they execute, they leave the traces around: what have I tried? Was I successful? What was the outcome? The execution traces. Those three pillars. Okay, so let's look at that in the context of our bank account opening agent. This is a simplified view, but you can see this graph here. It has a combination of business concepts like checks and accounts and credit history and stuff like that. This is a process-following agent, or a process-guided agent. We want this type of agent to actually follow a process. We've also encoded that in the ontology, a business process. And then if you look at the node that is surrounded by green, the check compliance one, we flip to the technical ontology, and we've put in the graph here, we've discovered and encoded that in order to do a compliance check, you might imagine that you need to resolve a government-issued ID. And then we say that in this particular organization, there are two data sources that can help us with that. It's the motor vehicle records and the passport verification one. That's really great. So then when our agents come in here and they realize I'm going to check compliance, I need a government-issued ID, here are the two ways that I can resolve that. When they execute and they try that, they leave the third pillar, the execution traces for that. And they're more sophisticated than what's on this simplified slide, but it involves things like, okay, where was I, what did I do, what is my context, and was I successful? And ultimately it leads out to some kind of a score. And you use that as input. It's like, okay, I've been very successful using the DMV lookup, for example, then I'm more likely to choose one if I'm in the right context in my next invocation. Three pillars of the ontology-based semantic layer: a business ontology, a technical ontology, the execution traces. Taken together, they solve all four of the problems. We now have a very easy way to discover the data sources. We know if they're trustworthy or not. We know that top-down by some kind of human-curated knowledge, right, an administrator of some sort saying it. We also know it bottom-up through the execution traces. This is what actually worked in reality, in practice. We have a single governed place that maps business intent and the concepts to those data sources, so we don't repeat ourselves. If something changes, that cascades across all my agents, right? And we have self-learning. So my agent that wakes up tomorrow is slightly smarter than it was today, and not just self-learning on an individual agent, but across agents as well. So we are moving from this world, a world of thick agents with manually wired data sources, into this world, where we have thin agents on a smarter shared ontology-based semantic layer. And this allows us to do a ton more agents without having to re-engineer them every time. Thin agents on top of a smarter shared substrate. If you think this is interesting, there's documentation on a web page that outlines more information about this, if you see the QR code here. You can also come and talk to us at the booth. We have the big booth here at the Expo, P3. We love talking about this kind of stuff. But not just that, this is one pattern, a very exciting pattern that we see a lot of traction around right now for using graphs in AI. But there are hundreds of more interesting patterns that combine graphs and AI. Ten of them are actually in the graph track that is kicking off right now in room 2005. And you have some really amazing talks from organizations like the Gates Foundation, Monday.com, JP Morgan Chase, Berkeley, New York Times, and so on and so forth. So go check out that thing. And then finally, this was primarily centered around organizations where you deal with many data sources and many agents. But if you are a startup building on Neo4j, love you. There is a startup program for Neo4j that is phenomenal. You get access to free credit, but more importantly, we built up a dedicated solution engineering team that spends every day working with startups for free, helping them model their data in Neo4j, tune it for performance, and so on and so forth. So please sign up for our startup program. Thank you very much. Enjoy the conference. Have a good day, everyone. work at a start-up, and you have one application that sits on top of one Postgres database, that's not hard. The data is in that postgres database. But in an enterprise ecosystem, you don't have one database, you have a hundred databases, and you have Snowflake and Databricks, probably, and you have S3 buckets, and so on and so forth. You have to do that work manually from scratch every single time. And then, when you've found the data sources, you know, in an enterprise, there's lots of duplication of data, so they need to figure out, like, is this the right data? Is it the right version? Can I trust it? Am I allowed to access it? So on and so forth. It also violates one of the core principles of software engineering, the dry principle, don't repeat yourself. So when something changes, that cascades across all of your agents. You have to kind of manually rewire all of them all the time, which works, but it's just a lot of work. And then finally, there's no learning around the data sources and how your agents operate on them. So when your agent wakes up tomorrow, it's not smarter than it was today, and there certainly isn't any cross-agent learning, because all of that wiring between business intent and the data sources is encoded in a combination of code and problems. So I know what you're all thinking. Markdown files, skills to the rescue! And yes and no. You can come talk to me afterwards for kind of the full version of this, but we've seen a ton of team that tried to solve this problem using just markdown files. And the summary is, it is part of the solution, but it is not the solution. But don't take it from me, take it from Soix. A week ago on the Latent Space podcast, I said, hey, guys, you've got to learn your databases. You cannot vibe code with just markdown files. So we've been solving this problem at scale for some really massive organizations recently, including a Fortune 20 global bank, a massive tech platform company based here in the Bay area, and a leading fintech company. And the pattern that is emerging is that in order to do agents at scale, we need thin agents on a smarter shared substrate. Thin agents on a smarter shared substrate. And what does that look like in practice? There are three pillars to that. The first pillar is a business facing ontology. And the word ontology, like I grew up in this world, people talked about ontology forever. More recently, it's become very hype, probably thanks to Palantir, but also the rise of AI. And there's a lot of people who want to make ontology really complex. But the core concepts are actually super simple. What are the key concepts in your organization? What are the key things that you can do to make a business organization? In our banking example, customers, accounts, debit cards, checks, transactions, and how do they all relate? But very importantly, they are expressed in a way that makes sense to all the human beings working in your universe, right? All the people working in your company, it's expressed in that way. In other words, you don't say F underscore name. No, you have a customer and they have a first name. So that's the first, a business facing ontology. The second pillar is a technical ontology. This is all the metadata of all the data sources and data assets in your enterprise ecosystem. I have 14 Oracle databases, I have 15 Neo4j databases, I have Snowflake and Databricks, and I have S3 buckets, and all of that kind of stuff. Where do they sit? What are the schemas? All of that kind of good stuff. You construct that technical ontology in three key ways that we can talk about later, though not in this talk. And then you have a mapping between the two. So that customer that has a first name, that first name has a system of record, and over there, there's an Oracle database with a column called F underscore name, the mapping between the two. And then the third pillar is the runtime signals out of your agents. When they walk this graph and they execute, they leave the traces around, what have I tried? Was I successful? What was the outcome? The execution traces. Those three pillars. Okay, so let's look at that in the context of our bank account opening agent. This is a simplified view, but you can see this graph here, it has a combination of business concepts like checks and accounts and credit history and stuff like that. This is a process following agent, or a process guided agent. We want this type of agent to actually follow a process. We've also encoded that in the ontology, a business process. And then if you look at the node that is surrounded by green, the check compliance one, we flip to the technical ontology, and we've put in the graph here, we've discovered and encoded that in order to do a compliance check, you might imagine that you need to resolve a government-issued ID. And then we say that in this particular organization, there are two data sources that can help us with that. It's the motor vehicle records and the passport verification one. That's really great. So then when our agents come in here and they realize I'm going to check compliance, I need a government-issued ID, here are the two ways that I can resolve that. When they execute and they try that, they leave the third pillar, the execution traces for that. And they're more sophisticated than what's on this simplified slide, but it involves things like, okay, where was I, what did I do, what is my context, and was I successful? And ultimately it leads out to some kind of a score. And you use that as input. It's like, okay, I've been very successful using the DMV lookup, for example, then I'm more likely to choose one if I'm in the right context in my next invocation. Three pillars of the ontology-based semantic layer, a business ontology, a technical ontology, the execution traces taken together, they solve all four of the problems. We now have a very easy way to discover the data sources. We know if they're trustworthy or not. We know that top-down by some kind of human curated knowledge, right, an administrator of some sort saying it. We also know it bottom-up through the execution traces. This is what actually worked in reality, in practice. We have a single governed place that maps business intent and the concepts to those data sources, so we don't repeat ourselves if something changes that cascades across all my agents, right? And we have self learning. So my agent that wakes up tomorrow is slightly smarter than it was today, and not just self learning on an individual agent, but across agents as well. So we are moving from this world, a world of thick agents with manually wired data sources, into this world, where we have thin agents on a smarter shared ontology-based semantic layer. And this allows us to do a ton more agents without having to re-engineer them every time. Thin agents on top of a smarter shared substrate. If you think this is interesting, there's a documentation on a web page that outlines more information about this, if you see the QR code here. You can also come and talk to us at the the booth, we have the big booth here at the Expo, P3. We love talking about this kind of stuff. But not just that, this is one pattern, a very exciting pattern that we see a lot of traction around right now for using graphs in AI. But there's hundreds of more interesting patterns that combines graphs and AI. Ten of them is actually in the graph track that is kicking off right now in room 2005. And you have some really amazing talks from organizations like the Gates Foundation, Monday.com, JP Morgan Chase, Berkeley, New York Times, and so on and so forth. So go check out that thing. And then finally, this was primarily centered around organizations where you deal with many data sources and many agents. But if you are a start-up building on Neo4j, love you. There is a start-up program for Neo4j that is phenomenal. You get access to free credit, but more importantly, we built up a data dedicated solution engineering team that is a dedicated solution engineering team that spend every day working with startups for free, helping them model their data in Neo4j, tune it for performance, and so on and so forth. So please sign up for our start-up program. Thank you very much. Enjoy the conference. Have a good day, everyone.