AI Engineer

Notion's Token Town — Sarah Sachs, Notion

1886 summary words 8 min summary Watch video

Start with the signal

8 min read

Summary

At-a-Glance

Verdict: Watch fully
Core thesis: AI product companies avoid becoming "token-poor" by treating model access as a volatile commodity: preserve multi-provider optionality, route work by end-to-end cost/capability/latency, use open weights and deterministic software where possible, and make security and orchestration core product capabilities.
Why it matters: This is a practical operating thesis for building agent systems and software factories without surrendering margin, product quality, or security to a single frontier-model vendor.
Best use: Use it as an architecture and vendor-strategy briefing for model routing, agent orchestration, governance, and the economics of AI-enabled workflows.

Executive Summary

Sarah Sachs, who leads Notion's AI engineering teams, argues that the primary applied-AI challenge is not simply gaining access to the strongest model. It is building a sustainable product when model suppliers are also potential competitors, model pricing changes unpredictably, and nominally similar upgrades can radically increase token consumption or price. Her prescription is to compete on customer understanding, data flywheels, workflow orchestration, and interface quality—not on reselling a lab's tokens.

Her central operating model is model optionality. Notion maintains multiple model paths, with its Auto Model handling roughly 75% of traffic while preserving access to state-of-the-art models when a task genuinely requires them. The decision unit should not be token price or one-call latency alone, but the end-to-end trajectory: required capability, tool failures, latency, total cost, and user outcome. This also creates negotiating leverage because the company can credibly move traffic rather than accept vendor lock-in in return for discounts.

Sachs separates frontier work from routine work. Expensive frontier models may be justified for difficult tasks such as large-scale data analysis, but routine inbox triage should not automatically receive the most capable model. Open-weight models increasingly cover moderate tasks and can be improved with reinforcement learning; deterministic jobs should bypass LLMs entirely through CPUs, CLIs, workers, SQL, or ordinary code. This reduces cost while improving determinism.

The talk closes by arguing that the next constraint on agentic systems is security and orchestration. Systems become dangerous when they combine private data, untrusted content, and external communication capability—the "lethal trifecta." Notion's software-factory example places agents inside a collaborative, inspectable work surface: one agent scopes work, humans provide missing context, specialist agents provide customer insight, coding agents create PRs, and another model reviews them. The intended result is an auditable multi-agent workflow rather than an autonomous system engineers must continuously babysit.

Key Takeaways

Claim: Treat frontier-model vendors as strategically misaligned suppliers, because their first-party products can receive better economics than your resale-based AI product. | Evidence: Sachs describes recurring real-world scenarios in which a model upgrade keeps per-token pricing flat but consumes 3x more output tokens, or raises price by 40% while the prior model is deprecated within four months. She also cites a SemiAnalysis comparison showing major differences between frontier labs' first-party subscription economics and API pricing. | Implication: Do not base unit economics or product positioning on a single vendor's current API pricing; maintain the ability to substitute models and build differentiated value above the token layer. | Caveat: The speaker's "supplier is your competitor" framing is a strategic heuristic, not proof that every model provider or commercial agreement is adversarial.
Claim: Model routing should optimize cost per capability per second across a full workflow trajectory, not benchmark rank, raw token price, or latency for an isolated call. | Evidence: Notion's partnership evaluation for Parallel web search considered complete search trajectories rather than a single call; Sachs says this granularity exposes trade-offs that one-call cost or latency obscures. She recommends measuring product-specific needs such as tool-error rates and actual latency requirements. | Implication: Build an eval and routing layer around end-to-end task success, cost, latency, tool reliability, and user impact before making provider or model-selection decisions. | Caveat: This requires proprietary, continuously maintained evaluations tied to real workflows; public benchmarks alone cannot supply the necessary routing judgment.
Claim: Use model optionality as both a product capability and commercial leverage, rather than accepting a large vendor discount in exchange for dependency. | Evidence: Notion's Auto Model handles about 75% of its traffic while customers retain access to multiple state-of-the-art models. Sachs advises building a multimodal harness that can switch models, and says Notion exchanges high-value use-case evaluations and early-access feedback with frontier labs instead of relying on extraordinarily large commitments. | Implication: Invest in a provider-neutral model abstraction and switching capability early; the cost of building it may be lower than the strategic and financial cost of being unable to leave a provider. | Caveat: Interoperability has genuine engineering costs, including challenges such as cache invalidation when switching models mid-transcript.
Claim: Frontier models should be reserved for genuinely hard work; moderate tasks are increasingly candidates for open-weight models, while deterministic tasks should not use LLMs at all. | Evidence: Sachs contrasts large-scale data analysis, where Notion may recommend Claude Opus, with email triage, where using Opus would be wasteful. She argues open-weight models now handle more than small SFT tasks and may close capability gaps over time; she also cites deterministic alternatives: convert CSV to PDF with code, use a CLI for tool calls, and run SQL directly rather than ask an LLM to do it. | Implication: Create a task taxonomy with at least three paths—frontier, open-weight/cheaper model, and deterministic execution—and progressively migrate tasks downward as capability improves. | Caveat: Open-weight suitability must be tested on the actual workflow; the argument is not that open-weight models are universally the strongest models.
Claim: The critical near-term agent problem is security, especially when autonomous systems combine sensitive context, untrusted inputs, and the ability to act externally. | Evidence: Sachs references Simon Willison's "lethal trifecta": access to private data, exposure to untrusted content through sources such as ingestion, MCP, or email, and the ability to communicate externally through actions, payloads, or web search. She stresses that more autonomy means more unsupervised risk. | Implication: Gate any agent that reads sensitive data and consumes external content before it can take outbound actions; isolate execution environments and explicitly define what each agent can see, retain, and do. | Caveat: The transcript names the risk pattern but does not provide a detailed control framework for authorization, sandboxing, auditability, or prompt-injection containment.
Claim: A useful software factory is a collaborative orchestration layer, not a collection of autonomous coding agents operating without context or review. | Evidence: Notion's demo starts with Claude scoping a task into an active collaborative document, asks a human team member to resolve a missing question, brings in Decagon for customer-voice data, has Claude create a PR, and asks Codex to review it, finding two issues. Sachs says Notion internally routes substantial feedback to appropriate teams and lets coding agents take the first step; she cites over three minutes saved per task for customers. | Implication: Design agent systems around inspectable state, specialist delegation, human escalation, and independent review—not merely around a single agent's ability to generate code. | Caveat: The stated time-saving figure is presented as a customer ROI claim without methodology, task distribution, or quality/error-rate data.

Detailed Brief

Notion's durable system-of-record thesis

Claims: Sachs frames AI adoption as a maturity path from thought partner, to assistant, to teammate, to a critical workflow system in which humans and agents collaborate with each other.; Her explanation for stalled AI transformation is fragmented data and the absence of a durable shared system of record.; She claims 88% of organizations have not progressed beyond AI used as an assistant.
Evidence: Early AI usage is described as copy-pasting text from ChatGPT into email; assistant-level AI executes isolated user-requested tasks, while teammate-level AI performs repeatable processes.; Notion positions its collaborative workspace as the place where human-human, human-agent, and agent-agent coordination can persist.
Caveats: The 88% figure is asserted without a source or definition of the maturity categories.; A shared workspace alone does not solve data quality, permissions, workflow ownership, or agent reliability.
Implications: The control plane for agentic work should preserve task context, decisions, artifacts, feedback, and ownership in a shared inspectable surface rather than scatter them across prompts and point tools.

Governance and persistence as product differentiation

Claims: Sachs treats governance as more than policy compliance: model optionality can improve customers' visibility into data usage, maintainability, and control.; She identifies persistent enterprise knowledge—what agents see, what they do, and what remains available afterward—as under-discussed in multi-agent design.; Sandboxes and computer-style execution environments can improve determinism as well as security and token efficiency.
Evidence: The presentation contrasts a factory engineers must babysit with a managed orchestration environment where work can be inspected and coordinated through a live collaborative document.
Caveats: The presentation does not specify data-retention policy, memory scoping, cross-agent permission inheritance, or audit-log implementation.
Implications: Enterprise agent platforms should make state persistence and governance deliberate configuration choices, with clear boundaries between ephemeral execution context, reusable organizational knowledge, and sensitive data.

Notable Concepts & Terms

Token-poor: Sachs's label for an AI business whose product economics become unsustainable because it overpays for inference or applies expensive models to work that does not require them.
Cost per capability per second: A routing lens that combines task success, capability, end-to-end latency, and total cost rather than optimizing token price alone.
Model optionality: The technical and commercial ability to switch models or providers, which protects product quality, limits vendor lock-in, and creates negotiating leverage.
Auto Model: Notion's routing approach, said to handle about 75% of traffic, that selects among models rather than binding all work to one provider.
Open-weight models: Models available with weights that can lower inference cost, create a credible alternative to frontier APIs, and increasingly address moderate-complexity tasks.
Lethal trifecta: A security-risk pattern attributed to Simon Willison: private-data access plus untrusted content plus external communication/action capability.
Software factory: An orchestrated workflow in which agents and people jointly scope, enrich, implement, review, route, and close work rather than relying on one autonomous coding agent.
AI Switzerland: Notion's positioning as provider-neutral infrastructure that gives customers access to multiple leading models without tying them to a single lab.

Operator Notes / Why Ken Should Care

Require every production agent workflow to declare its model-routing policy, fallback model, maximum cost budget, expected latency target, and deterministic non-LLM path where applicable.
Create a product-specific evaluation suite that scores complete trajectories, including tool-call failures, retries, user-visible latency, cost, and task success—not just model benchmarks.
Audit agent workflows for the lethal trifecta; block or require explicit approval for combinations of sensitive data access, untrusted ingestion, and outbound action.
Separate durable organizational memory from per-task context, and define who can inspect, modify, retain, and reuse each category of agent state.
Avoid long-term volume commitments or architectural dependencies that prevent provider switching unless the commercial benefit demonstrably exceeds the cost of lost optionality.
Use a staged multi-agent delivery pattern for software work: task specification, human clarification where needed, specialist evidence gathering, implementation, independent review, and monitored closure.

Source/Metadata

Title: Notion's Token Town — Sarah Sachs, Notion
Transcript words: 6360
Duration seconds: 1435
Timestamp note: No usable timestamps or chapter markers were present. The supplied transcript contains substantial repeated sections and trailing extraction noise.

Search this video Resume transcript at 0%

Full transcript 4119 words · 28 min read

0:02