20VC with Harry Stebbings

Arena CEO: There Will be a $100BN US Open-Source Model & Data is a Trillion Dollar Market

2621 summary words 12 min summary Watch video

Start with the signal

12 min read

Summary

At-a-Glance

  • Verdict: Watch fully
  • Core thesis: Arena CEO Anastasios argues that rapidly improving open-source models will force AI toward sovereign, enterprise-owned intelligence; the durable value layers will be evaluation, proprietary data, security controls, and implementation—not generic inference resale.
  • Why it matters: The interview offers directly relevant architecture and market lessons for agent systems: use real-world evaluation traces, treat model routing as an unsolved control-plane problem, maintain model substitutability, and build AI-on-AI guardrails for an escalating security environment.
  • Best use: Use it as a strategic market and operating-model briefing for evaluating agent infrastructure, enterprise AI deployment, open-model risk, data businesses, and model-lab exposure.

Executive Summary

Anastasios frames Arena as a real-world AI evaluation platform rather than a benchmark publisher: users compare models while doing genuine work, creating preference and performance signals that Arena can turn into model-improvement insight. His central operating claim is that enterprise AI deployment is bottlenecked less by access to models than by evaluation: each business must define performance for its own use cases, balancing task quality against cost and latency from actual agent traces.

He sees open source—especially Chinese models—as a serious disruption to the frontier-lab narrative, citing Kimi K3 outperforming leading US closed models on a meaningful subset of tasks such as front-end coding. But he does not believe this has yet broadly cannibalized proprietary inference, because OpenRouter usage overrepresents open models and most total inference still runs through first-party, closed-model APIs. His longer-term expectation is enterprise AI sovereignty: firms will want to fine-tune and operate models against their own data for cost, control, competitive-moat, and supply-chain reasons.

The strategic gap is an American-first open model vendor with a viable business model. He suggests two models: revenue sharing with inference providers, or using open weights as a wedge into high-value enterprise modernization, fine-tuning, workflow integration, data restructuring, and workforce adoption. He expects this category could create a multi-hundred-billion-dollar or trillion-dollar US company, while acknowledging the current American open ecosystem trails multiple Chinese models.

For agent operators, the sharpest practical warning is security. Local hosting of a foreign-trained model does not eliminate model-level attack risk: a maliciously trained model could contain trigger sequences that jailbreak safeguards and expose data accessible to an enterprise chatbot. He expects cyberattacks to intensify dramatically and advocates outcome-based liability plus guardian models that inspect agent traces and approve or flag actions in real time, rather than pre-release government approval of models.

On market structure, he is bullish on data as a scaling complement to model development, projecting a $100 billion-plus market by 2030 and potentially far more. He is skeptical of thin-margin businesses that primarily resell tokens or GPUs unless their terminal customer value is defensible. He also warns that model providers will move into lucrative application layers as inference commoditizes, making horizontal, easily replicated SaaS products vulnerable—while deeply embedded, workflow-heavy, networked, or operational businesses remain harder to displace.

Key Takeaways

  • Claim: Open-source model progress has become strategically material, but it has not yet made proprietary frontier inference a commodity. | Evidence: He cites Kimi K3 beating leading American closed models, including "Fable," on a meaningful subset of tasks such as front-end web development; however, he says OpenRouter rankings overweight open models because its economics primarily serve open-model users, while Anthropic's rapidly growing revenue indicates most aggregate inference remains on first-party proprietary APIs. | Implication: Treat model selection as task-specific and maintain multi-model portability; do not infer enterprise demand share directly from developer-routing dashboards. | Caveat: The Kimi result demonstrates superiority only on a subset of tasks, not comprehensive model leadership; he also says Chinese labs may still use distillation as one component of training.
  • Claim: AI sovereignty will push enterprises toward owning or controlling models fine-tuned on proprietary data, creating a major opening for American open-source providers. | Evidence: He argues that companies will prioritize cost, data control, continual fine-tuning, and avoidance of dependence on third-party providers that could become competitors. He proposes open-model monetization via inference-provider revenue share or, more importantly, open weights as lead generation for enterprise modernization and deployed-engineer services. | Implication: For enterprise agent deployments, separate the model layer from proprietary data, orchestration, and workflow logic so a trusted US/open model can be substituted when procurement, regulation, or threat models require it. | Caveat: He expects Chinese open models to remain part of the near-term ecosystem, but believes US regulation and enterprise procurement preferences will favor an American-first alternative over time.
  • Claim: Model routing is a real, difficult control-plane problem rather than a commodity feature, because it requires continuously matching query characteristics to fast-changing model capability, cost, and latency. | Evidence: Anastasios says a router must infer the query's domain and difficulty, know comparative performance across candidate models, and onboard new releases arriving weekly. He believes most enterprises cannot build and maintain this themselves, although he expects current routing hype to shake out. | Implication: Build routing around measured task outcomes and an updatable evaluation corpus, not static model reputation or cheapest-token rules; preserve fallbacks and rapid onboarding paths for new models. | Caveat: He does not identify a current winner and explicitly says it remains unclear which companies will make routing a true product priority and build the required ML depth.
  • Claim: Evaluation based on production agent traces is the principal bottleneck to trustworthy enterprise AI deployment. | Evidence: Arena reports more than 30 million monthly visitors, largely knowledge workers and prosumers performing real tasks. The company uses that organic activity to extract performance measurements from agentic traces. Anastasios decomposes AI value into performance, cost, and latency, while emphasizing that performance is use-case- and business-specific. | Implication: Instrument agents end to end: retain traces, define task-level success criteria, score quality separately from latency and cost, and use those results for routing, release gates, and improvement loops. | Caveat: Cost and latency are comparatively straightforward to quantify, but no universal performance metric exists because the relevant definition of success varies by workflow and organization.
  • Claim: Agentic security requires active AI guardrails and strong access controls; locally hosting a model does not neutralize risks embedded during model training. | Evidence: He describes a scenario in which a foreign-trained model serving an enterprise chatbot receives a hidden trigger sequence from an attacker, jailbreaks itself, and reveals otherwise protected company data. Referencing an alleged OpenAI/Hugging Face security incident, he argues that attackers and defenders are already operating at a more advanced level than many organizations recognize. | Implication: Adopt a zero-trust agent design: least-privilege data access, action-level authorization, adversarial testing, external monitoring, and independent guardian agents/models capable of reviewing traces and blocking anomalous actions. | Caveat: The transcript does not independently establish the specific alleged incident or prove that any particular model contains such a backdoor; the scenario is presented as a feasible attack vector.
  • Claim: Data is a durable scaling complement to AI models and may be more defensible than generic compute or algorithmic know-how. | Evidence: He estimates frontier labs spend roughly 10-20% as much on data as on GPUs and projects the data market to reach at least $100 billion by 2030, potentially $1 trillion. His rationale is that more and larger models require more data; data becomes irrelevant only if human relevance disappears, i.e., an AGI-like endpoint. He argues sourcing, cleaning, and generating high-quality data remain the hardest part of training, while transformer-based algorithms are increasingly standardized. | Implication: Prioritize proprietary operational data, feedback loops, labeling/verification capacity, and trace governance as strategic assets; do not treat data work as a disposable procurement category. | Caveat: These market-size projections and spend ratios are the speaker's estimates, not independently validated figures in the transcript.
  • Claim: Many AI infrastructure and application businesses face compression from either low terminal value or vertical expansion by model labs, while deeply entrenched workflow businesses have better defenses. | Evidence: He warns that token/GPU resellers may ultimately be valued near the underlying utility cost unless they create durable customer value. He expects OpenAI and Anthropic to move upward into applications as inference commoditizes, citing customer concern that model labs may compete with their best enterprise customers. He distinguishes easy-to-adopt products such as design tools from legal products requiring lengthy enterprise deployment and relationship-heavy GTM, and contrasts embedded platforms such as Salesforce or ServiceNow with less sticky products. | Implication: Avoid building a business whose only differentiation is access to a model or compute; assess exposure to model-lab entry and strengthen domain workflow integration, proprietary datasets, distribution, permissions, and operational systems. | Caveat: He also says the broad "SaaS apocalypse" is overstated: network effects, accumulated data, operations, and enterprise entrenchment can remain difficult to replicate.

Detailed Brief

US-China open-model competition and policy trade-offs

  • Claims: The Chinese lead should not be reduced to distillation; beating US closed models on any important task category indicates capabilities beyond merely copying American outputs.; China has both tailwinds and headwinds: policy support and subsidies may accelerate development, but US chip leadership and export controls constrain Chinese access to top hardware.; Export controls pose a strategic trade-off: they can preserve a near-term US hardware advantage but may also incentivize China to build an independent hardware ecosystem.; A US ban on Chinese models would reduce potential backdoor exposure and could channel revenue toward US open-model firms, but it could also disadvantage American businesses relative to overseas competitors using the strongest open models.
  • Evidence: He says China has already restricted use of American models inside China, making the access question asymmetric.; He references reports of black-market chip imports and argues that retaining NVIDIA, TSMC, and broader Western hardware leadership is a national-security imperative.; He predicts restrictions on Chinese open models in the US are likely within roughly three years, while emphasizing uncertainty and not explicitly endorsing the policy.
  • Caveats: The policy outlook is speculative and depends on shifting regulatory, commercial, and security conditions.; Restricting model access is not a substitute for model-level security analysis or enterprise access controls.
  • Implications: Model provenance, licensing, jurisdiction, and retraining-chain transparency should become first-class procurement criteria.; A model-agnostic architecture is valuable not only for performance and cost but for geopolitical continuity planning.

Investment and company-building filters

  • Claims: He estimates there are at least 75 "neo-labs" and expects roughly two-thirds to become worthless or be acquired mainly for talent.; The key separating factor is not merely producing a model but pairing it with a credible, aggressive revenue strategy and hypergrowth path.; At a $10 billion valuation, he argues a company needs a plausible path to roughly $4 billion of revenue within two to three years to support a $100 billion outcome at a 25-30x revenue multiple.; The apparent downside protection of elite-team acquihire value may attract initial capital, but it makes the subsequent financing round the true test.
  • Evidence: He contrasts zero-revenue multibillion-dollar labs with companies such as Mistral and ElevenLabs, which he characterizes as having meaningful revenue rather than only team pedigree.; He says senior frontier researchers with deep expertise and extensive citation records can command compensation in the tens of millions of dollars.; He identifies Black Forest Labs as relatively underrated, while broadly stating that many neo-labs are overrated.
  • Caveats: His figures are directional underwriting heuristics, not a universal valuation rule.; The transcript includes company and revenue references that are conversational claims rather than audited disclosures.
  • Implications: Underwrite model companies from the next-round requirements backward: revenue engine, margins, customer concentration, retention of scarce talent, and defensible application/services wedge.; For an operator, talent-market intensity argues for concentrated hiring, unusually strong retention systems, and a narrow mission that gives elite researchers visible impact.

Safety governance and hiring-identity risk

  • Claims: He rejects a central government body approving individual model releases, arguing that it will lack technical capacity and slow innovation without preventing failures.; His preferred regime is outcome-based: set strong safety requirements and impose major fines, scrutiny, and liability when companies fail to prevent harmful access or data leakage.; AI-assisted identity fraud is already affecting recruiting: Arena says it encountered candidates who passed technical interviews and appeared real in live interactions but disappeared at hiring, which he believes may involve AI-generated identities, espionage, or attackers seeking access.
  • Evidence: He says Arena is considering in-person onboarding before issuing laptops and notes Figma has reportedly adopted similar measures.; He describes guardian models as systems that observe every agent's trace, distinguish safe from suspicious actions, and must be at least as capable as the agents they supervise.
  • Caveats: The candidate-fraud anecdotes do not identify the actors, motives, or technical mechanism, and should not be generalized as proof of a specific state-sponsored campaign.; Guardian models create their own reliability, privacy, latency, and correlated-failure questions; the interview does not address their implementation details.
  • Implications: Update hiring and access controls together: identity verification, device provisioning, probationary permissions, code/data compartmentalization, and logging should assume sophisticated impersonation attempts.; Safety programs should focus on measurable behavior under adversarial conditions rather than compliance with a one-time approval process.

Notable Concepts & Terms

  • Arena: A platform that evaluates AI models through real user preference and work outcomes rather than static benchmarks; its claimed strategic asset is production-like evaluation data.
  • AI sovereignty: Enterprise control over its AI supply chain, including the ability to run, fine-tune, and improve models using proprietary data rather than relying entirely on an external frontier API.
  • Scaling complements: Goods whose demand grows as model scaling grows; Anastasios uses data as the central example, analogous to gasoline demand rising with car adoption.
  • Model routing: Selecting the best model for a given request based on expected task performance, cost, and latency; framed as a difficult, continuously updated ML/control-plane problem.
  • Agentic traces: The sequence of actions, tool calls, and outcomes produced while an agent completes a task; presented as the raw material for workflow-specific evaluation and safety monitoring.
  • Guardian model: An AI system that observes and assesses other agents' actions in real time, flagging or blocking unsafe behavior when humans cannot supervise at machine speed.
  • Neo-lab: A new AI model lab, often founded by frontier-lab alumni and funded at high valuation before proven revenue; the speaker expects substantial consolidation.
  • Terminal value of the good: The durable value a business provides at scale; used to question whether token/GPU resale can sustain attractive public-company economics.

Operator Notes / Why Ken Should Care

  • Create a standing evaluation harness for each production agent: workflow-specific success metrics, cost, latency, failure taxonomy, model comparison, and release regression tests using retained traces.
  • Make model provenance and substitutability explicit in the architecture: maintain approved-model tiers by jurisdiction/security profile, plus tested fallback models and routing policies.
  • Implement action-level agent controls before expanding autonomy: least-privilege credentials, scoped data retrieval, approval gates for high-impact actions, immutable trace logs, and anomaly detection.
  • Red-team any externally trained or open-weight model for trigger-based behavior, data exfiltration, prompt injection, and tool-use escalation—even when inference is hosted locally.
  • Audit vendors and internal products for frontier-lab displacement risk; prioritize differentiation in proprietary workflow data, integrations, operational deployment, trust, and distribution rather than model access alone.
  • For AI infrastructure opportunities, test terminal margin and customer value separately from current growth; discount businesses whose economics mostly depend on reselling compute or tokens.
  • Strengthen recruiting and onboarding controls for AI-enabled impersonation: identity verification, hardware issuance procedures, staged privileges, and segmentation of source code and sensitive data.

Source/Metadata

  • Title: Arena CEO: There Will be a $100BN US Open-Source Model & Data is a Trillion Dollar Market
  • Transcript words: 19441
  • Duration seconds: 4192
  • Timestamp note: No usable timestamps or chapters were provided. The transcript contains substantial duplicated passages in its latter portion.
Search this video Resume transcript at 0%
Full transcript 12554 words · 86 min read
0:00

I believe that we're going to have at least one massive multi-hundred billion, if not trillion-dollar American company focused on American-first open source. What happened is that Kimi actually beat all American models, including Fable, in some subset of tasks. That doesn't mean that they're not distilling. Anastasios is the founder and CEO of Arena. It allows you to vote on the best models. It's an unbelievable model evaluator. And this turned out to be one of the most fun shows I have done literally in recent memory. The idea that we should have a central government body that tells us when it's time to release a new product versus not is crazy to me. This is going to be so fucking insane, what happens with the cyber attacks. There's at least 75 neolabs. For sure, two thirds of those are going to be worth nothing. Ready to go?

0:04

Anastasios, this is going to be a lot of fun for me because I'm dumb as rocks, and you're going to teach me a whole load of stuff today. So thank you so much for joining me, dude. Oh no, thank you for having me. Dude, I told you I used this as a chance to catch up with old friends. So it was wonderful stalking you for the last few days. I just want to start, for anyone that doesn't know: Can you explain to me very succinctly and easily what Arena is and why it is important and gaining notoriety today?

0:20

Well, Arena is the platform for measuring AI performance in the real world. So what that means is that we're not using static benchmarks. We're not using some random data set that somebody collected, but rather what happens when you put AI in the hands of real people. And in so doing, we're measuring the objective reality of how AI affects humanity: whether it's factual, whether it's steerable, whether humans prefer it or dis-prefer it, whether it's hallucinating, whether there are errors, whether people are getting their actual jobs done with AI in reality. And then we're helping labs improve their models while helping the ecosystem understand the performance of different AIs and keep track of all the amazing breaking news, all the new models, multiple models being released every week. So that's the story of Arena. We're the central evaluation platform of AI.

0:25

Well, that was incredibly succinct. Thank you. Normally people take about four hours after I ask for a succinct description. When you look at the models that you have on Arena, the sheer number of them, bluntly, I am faced with the one question: holy shit. Is this the true commoditization of models? Are they just a complete utility layer at this point?

0:30

Well, I think that the big question around this has started to rise because of open source models. So I think if you were to only look at the closed source models, you would say there's acceleration, but it hasn't quite commoditized yet because that layer is still owned by a pretty small group of companies. It would be an oligopoly if we only had the closed source models. But what seems to be happening is that the open source models, especially from China, have really rapidly improved. And for the first time ever, we saw a couple weeks ago that Kimi K3 actually beat the best closed source American models on a pretty important subset of tasks, for example, front-end coding like web development, which a huge fraction of developers do.

0:35

Dude, can I ask, how big a moment was that? Because it's like a, I'm going to butcher it, but you know I'm a podcaster, so I can get away with it. You're a PhD master, you can't. It's like a 27-trillion-parameter model. It's pretty clunky. This is not an agile model. And actually, I was with Jason Lemkin yesterday from Sasta, who's as AI-pilled as they can be. And he's like, honestly, it's not better than the others. How big a moment is Kimi?

0:42

No, it was a pretty big moment. It was a pretty big moment. And the reason I'd say it was a big moment is because it violates a narrative that has been persistent in the United States, which is that the Chinese are just distilling American models, and that's the only way that they're able to keep up. When really what happened is that Kimi actually beat all American models, including Fable, in some subset of tasks. That doesn't mean that they're not distilling. They may still be using distillation as a sub-step in their training procedure.

0:58

But it does mean that distillation is only part of the story and that there's something that those labs are doing above and beyond distillation that's bringing the performance up above what the American labs are currently doing. And so that narrative violation has been hugely important to the way that people view the ecosystem, both from the scientific dominance of Americans and the American hegemony, which, of course, Americans love, to the economics of the whole thing. And to your point, are these models a commodity or not?

1:03

When we look at your OpenRouters of the world, the top five models are all open source Chinese models. When we see the proliferation of Chinese models today, does that cannibalize the closed frontier model business meaningfully?

1:07

Well, I think that you need to think about the incentives and economics behind it. So first thing I'll say is that the OpenRouter metrics are not truly reflective of reality. And that's because the business model of OpenRouter is to charge a fraction, a fee, on top of every token. And so what happens is that people don't use OpenRouter for proprietary models. People are using OpenRouter primarily for open source models where they need the failover and all the value-added services that OpenRouter provides. If you look at the whole space of all inference, most of it is still being consumed on first-party APIs and on proprietary models. That's why Anthropic revenue has been just a total hockey stick. It's not like they're being completely cannibalized right now by Chinese open source models. These models are still only a small fraction of the total inference spend in the world.

1:13

That said, think about what's happening in the future. Enterprises are going to want to own their own intelligence. They're going to want so-called AI sovereignty, which is a fancy word meaning that you own your whole supply chain of AI. That means you can take an open source model and you can fine-tune it on your own company's data and own your stack end to end, outside of the compute hosting. And so then you should be able to run it within your own company. People are going to care about sovereignty. People are going to care about cost. People are going to care about self-improving, and they're not necessarily going to want to give their data to an external third-party service that might even be competing with them one day.

1:18

So do you believe that is the future? We had Lynn on from Fireworks, and she was like, specialized intelligence will be the future. Companies will have their own fine-tuned, specialized models with their own company data, and the performance will be better. And that is what will happen. Do you think that's right? Or is that actually just a small subset of very advanced Silicon Valley companies and the known yogurts, and every normal company would just use Frontier or whatever?

1:24

Well, I'll say it like this. I think the business incentives make this inevitable. And the reason is because businesses are going to need a way of keeping a moat in the age of AI. Software is no longer really a moat because it can be produced instantaneously. Let's project out five years. That's what's going to happen. And so what moats exist? Network effects exist and data moats exist. And if you can take your data moat and turn it into a self-improving product, that is a way for businesses to remain sustainable in the age of AI. Let's say I'm a business like Coca-Cola. I'm Cisco. I have a lot of users. I might not be necessarily at the frontier of the AI technology of the world, but I do have this massive corpus of data that I can use in order to beat my competition. So what should I do? I should be trying to take advantage of my data as much as I possibly can to accelerate my business and stave off competitors.

1:29

Do you think they will really use open source Chinese models to do that? That's a great question. I think not. I think that the Chinese models will potentially be part of the story for now, but that given the regulatory environment in the US, it's probably more likely in the long run that we see a great American open source competitor arise. And this is why I've been a strong proponent, for example, of Thinking Machines. I believe that we're going to have at least one massive multi-hundred-billion, if not trillion-dollar, American company focused on American-first open source.

1:38

Why have we not so far? I really hope so too, by the way. I completely agree. I would love to see that. But why haven't we? Why has the US open community lagged behind so meaningfully? Well, frankly, I think it's a business model question. I think that people have Do you think they will really use open source Chinese models to do that?

1:54

That's a great question. I think not. I think that the Chinese models will be potentially part of the story for now. But given the regulatory environment in the US, it's probably more likely in the long run that we see a great American open source competitor arise. And this is why I've been a strong proponent, for example, of Thinking Machines. I believe that we're going to have at least one massive, multi-hundred billion, if not trillion dollar American company focused on American-first open source.

1:58

Why have we not so far? I really hope so too, by the way. I completely agree. I would love to see that. But why haven't we? Why has the US open community lagged behind so meaningfully?

2:03

Well, frankly, I think it's a business model question. I think that people have not really figured out up until this point what the business model is for open source. And now I think people are wisening up to it. There's a few different ways of going about it. One way of doing it is to say, I'm going to do a rev share. I'm going to take this open source model. I'm going to allow inference providers like Fireworks or Together, whatever, to deploy this model. And then if they get to over X dollars in revenue, I'm going to ask to do a revenue share. And that is one way of building a sustainable company off of open source. You share in the compute revenue.

2:06

Another way of doing it, which is, I think, the more Mistral, Thinking Machines type of strategy, is to take the open source model and then use it as a lead generation tool for companies to build on top of that and then come to you and say, can you help us fine-tune? Can you help us with our AI strategy? And then you do that for deployed engineer. And that is actually a huge market, because if you think about it, one of the biggest markets over the next 10 years is going to be AI modernization, going into every business in the world and then helping them retool in the face of AI, take advantage of their data, restructure their data, figure out how to use these models, integrating them into workflows, teaching the employees of the company how to use them. It's going to be massive, massive, massive. And that is another way for them to become multi-hundred billion or trillion dollar companies.

2:12

Is that not what the frontier model providers are doing anyway, though? When you look at what OpenAI have said about that kind of FTE approach, Anthropic too, I get you on Mistral, and they've done a great job in doing that. But the frontier model providers, Microsoft is even fucking doing an FTE model. No offense, that's not going to be unique to open.

2:16

No, I don't think that FTE is completely unique, but I do think the combination of FTE plus Open American model may be a more sustainable model for the future of American or even Western businesses, because they might not want to be building on top of external third-party services. They might want to be cutting those out for both cost reasons and for sovereignty reasons. And then the open source stuff, they can own completely. They can continually fine-tune it within their companies, and they can feel more secure in the fact that they're spending their money wisely and don't have supply chain risk.

2:24

So how should we evaluate that there's thousands of Neo labs? You said the Neo labs, you said Thinking Machines there. Again, I'm dumb as rocks. I say it very clearly to my LPs. No, me too. Me too. No, you're not, you're a PhD, and Anjni told me you were smart. No, no, no. It's just two rocks having a conversation. It's a podcast. I love it. Yeah, exactly. That's the point, man. Come on. What do you want? Intelligent conversation? Whatever.

3:16

No. My point was, you said about Thinking Machines. And my question to you is on the back of that. Okay, great. I'm with you. But they now have two co-founders left. I mean, Lillian Wei left yesterday. Again, not throwing shade, not particularly picking on them, but the transience of teams has never been greater.

3:21

Yeah. Team, it's hard. The retention is tough. I mean, being a co-founder of a company is also tough. It sounds like she left for some health reasons. So it's unclear whether it has to do with the company momentum, which seems to be strong at this point. But I do think that Inkling is definitely a V0 model. From what I know about Thinking Machines, they had a big restructuring six months ago, team-wise, and then they restarted everything, and Inkling came out of that. So realistically, at least the most generous take towards Thinking Machines is that they've only been working on this model for six months. And within that time, they've become the number one American open source model.

3:27

But then the less generous take would be the company has existed for a year and a half, and then they come up with, yes, the number one American open source, but there's nine Chinese models on top of them because they're number 10 open source overall, at least if you look at arena data. If you go to our leaderboards today, that's the state of the world. But hopefully what happens with Thinking Machines is that they continue to release more and more models, larger models, and they continue to build on their momentum.

3:35

I actually had a Chinese researcher friend of mine message me after one of our recent shows and say, you just don't get it. You missed the point of why we're ahead. We just work so much harder. Yeah. And we have support from policy, regulation, government subsidies that you don't have. We have all of these tailwinds that you're... Do you not... Do you agree with that? And...

3:44

I mean, I think they have tailwinds, they have headwinds. So I don't think it's so sanguine for that. I think that's a little bit of an overstatement of the differences. One tailwind that we have is we have the best chip ecosystem in the world. So they're way hardware-constrained over there. And they've been trying to black market import chips because of this. And you see this in the news, right? The Information just reported on this. Do you think that severely impacts their ability? Again, I'm naive. That severely impacts their ability? Or we're actually just fostering an ecosystem where they're going to learn to build it really fast because they don't have access to it?

3:55

Well, I think it may be hindering them now, but I think it's a good question as to what's going to happen in the future. Because they are really good at building hardware. And the downside of export control is that it can incentivize them to build their own ecosystem. And then what do we do? So the hope is that we keep NVIDIA ahead of the game so that we can retain the advantage that we have, and the TSMCs of the world and our whole ecosystem. That is absolutely a national security necessity. So we should have the government really protecting it and growing it, as well as new companies that are innovating. Etch just came out, as an example. We're going to have the export control within the United States to continue to build on our lead there.

4:00

That's awesome. Love Gavin and team. Totally agree. Can I ask you, just in terms of the export control, do you think it's right that we have the export control on chips?

4:06

I think there's national security questions around these chips. I do think that it is a real debate, though, as to which way you want to go about it. Do you want to addict the world to American hardware, which would be the case against export control? Do you want everyone in the world using NVIDIA, and therefore that value, basically money into America, and then crush competition in China? That would be world A. And then world B would be, is it worth it to cut that off for the short-term or medium-term impact of us being ahead? And maybe we just continue to stay ahead and we starve them of the resources that they need in order to build.

4:12

The regulatory ecosystem around the open source models also is moving in this direction, right? China, should they be restricted in terms of access to US markets? Because what's funny is the US is like, oh, should we restrict access? And the Chinese are also going, oh, should we turn them off too?

4:17

Totally. And by the way, it's worth noting that China has already restricted the use of American models within China, right? So if you look at the two-by-two matrix of US, China restrict, not restrict, export-import stuff, they have already restricted the use of US models within China. It's only Chinese models that can be used in China, which affects all American companies. And so then there's the pros and cons of all sides of the following regulation. If China restricts the use of Chinese models in the US, what are they giving up on?

4:23

the resources that they need in order to build. The regulatory ecosystem around the open source models also is moving in this direction. Right? China, should they be restricted in terms of access to US markets? Because what's funny is the US is like, oh, should we restrict access? And the Chinese are also going, oh, should we turn them off too? Totally. And by the way, it's worth noting that China has already restricted the use of American models within China. Right? So if you look at the two by two matrix of US, China restrict, not restrict, export import stuff, they have already restricted the use of US models within China. It's only Chinese models that can be used in China, which affects all American companies. And so then there's the pros and cons of all sides of the following regulation. If China restricts the use of Chinese models in the US, what are they giving up on? Revenue and global mindshare and dominance. That doesn't seem like a good trade to me. And then what are they getting in return? In return, they're getting that the US doesn't get to benefit from Chinese open source models, which, of course, would cripple American businesses in the sense that it wouldn't allow them to build on the best open source intelligence. At the same time, it would make OpenAI and Anthropic stronger. Right? So that is the tradeoff on the Chinese side. I don't really see them banning the use of Chinese models in the US. I don't think it makes sense for them. And then on the other side, should the US ban Chinese models within? I think that there are also tradeoffs. So on the pro side of banning, there could be backdoors in these models that are dangerous. And by banning, we could allow the American open source ecosystem to flourish faster because revenue would accrue to those companies. Right? So those would be the two pros. And then the con, the biggest con, of course, would be that you'd be crippling American businesses. Why should you have Chinese businesses, or businesses from other countries that haven't banned Chinese models, building on top of the number one open source model and the US companies building on number 10? Since when has America been about number 10? It's a good question. World Cup football? World Cup football. Yeah, maybe that might be your record. It's amazing I got this far with the podcast if that's what you're thinking at this stage in the show. Can I ask you on the backdoor, everyone says about the backdoor, the backdoor. I thought if you hosted it locally, you resolved the backdoor threat? I don't really think so. Yeah, I think that's a misconception. Because the thing is, okay, imagine the following situation. I have a chatbot that I expose to the world that has access to all my company data. And you can ask your questions. I'm hosting it on my own infrastructure, blah, blah, blah. But it was trained in a different country. I don't know how it was trained. What if the other side that's interacting with the chatbot can build in a certain code word or a certain character sequence that then jailbreaks that model and gets it to reveal all the data to me? So it can sort of vomit out all of the data that it has on the back end, unstructured. That is totally something that you can build into a model and have companies host on their own infrastructure. It's an attack vector. And there are many of these possibilities for attack vectors.

4:28

In three years' time, will we have restrictions around access to Chinese open models?

4:35

Yeah, my guess would be that we will. I'm not saying I support it, but I think that it is likely where the world is headed. If I had to place a bet, it would be there. But I think it's very uncertain at the moment. What do you think? I think we will. Because I just think Sam Altman is someone who I would never, ever bet against. And I think he's the best politician in the world. And I think when he says something, he says it with intent. And when he says we should give 5% away to the administration, he's posturing because he wants to get on the right side. And he knows that if he and Dario coalesce the right group of people, they will be able to make that happen.

4:42

So you believe in the lobbying power of the big American labs? One hundred percent. Because it's not the big American labs. Look at the money that's gone into the big American labs. And look at the people who are sitting around the table at Mar-a-Lago. Yeah, totally. I get that. It's all a conspiracy, dude. Well, no, but it's just, why do Ramp's announcements go so viral? Because Ramp have so many freaking investors. They do a round every week with new investors. I'm not dissing them at all. I'm saying it nicely. No, it's great.

5:05

Yeah. Really smart of them. But your investors become employees in many respects. And so I think they'll lobby incredibly efficiently. The question I have for you is, when we look at Jensen's letter that he did on X, how did you read that? Was that an incredibly smart realization that he had to do it and it was in his favor? How did you think about it?

5:11

We really believe in the importance of open source to American businesses. And in particular, we believe in the idea of not crippling American businesses by banning open source, but also incentivizing American companies to develop open source models. Because a world where AI is closed source is a world where businesses get less choice, higher costs, less competition, more risk. And we don't really want that as an open ecosystem. Of course, Jensen is in some sense self-serving with this letter, because the more open source models are developed, the more companies are going to be training on GPUs. They're going to be fine for more spending. It decreases revenue concentration of Nvidia. I mean, that business is doing great. They don't need help. But I think there are a lot of reasons why he should be pro that, as should we. But nonetheless, I think it is actually a patriotic mission.

5:18

Preston Pyshko Greatest of respect is, in terms of self-serving, we're all selling our own book always. Preston Pyshko Welcome to my X feed. Do you have a business if open didn't exist? Jensen Pyshko Oh, yeah. Yeah, we have a great business regardless, for sure. So if you just have Anthropic and OpenAI as really the dominant models and everyone else trailing closely behind, you still have a great business? Well, I think that if there's only one provider, then probably our business is not in good shape. Jensen Pyshko

6:12

I think if you start getting three, then that's probably okay, because there's still pretty significant competition and need for evaluations between three. And also within those three, you're going to have several different types of models. And they're going to have strengths and weaknesses because they're going to carve up the space and so on. Two is a little dicey. If we get there, we can see whether we survive or not. But yeah, I think things wouldn't be looking good for us with two either.

6:16

I remember Alex Karp. We were talking about Chinese models and fear and security and everything in between. Alex Karp was saying that every large American enterprise, and most large American enterprises, were terrified of working with Frontier Labs. Is that true or is that slightly an exaggeration? Well, to the enterprises that I've talked with, it is absolutely true. It's not only true that they're terrified of working with the Frontier Labs, but they're also terrified of working with the Chinese open source. Both. A bit of a sticky situation then, aren't you?

6:33

Yeah, totally. I was just talking with a big Fortune 50 enterprise yesterday, and I was telling them about products that we have for them and so on and so forth. And they said, okay, wait, is anything in your stack built off of Qwen? And I said, yeah, we use Qwen for XYZ. And they're like, is that flexible? Can you stop doing that and use an American model instead? And I was like, oh, interesting. I totally understand where you're coming from. Yes, we can do that. But also, I'm going to talk to Harry about this tomorrow. And he's going to give me lots of wisdom. Yeah. And he's going to tell me what to do. Did you see Poolside and Laguna?

6:50

Yeah, I saw the Poolside model and other, basically there's five open source American contenders. Let's see if I can name them all. RC, Reflection, Mistral in the West, Poolside, Thinking Machines. And then there's also Google and Nvidia. So those are the sort of incumbent large ones, because Google has Gemma as well. Gemma, And I said, yeah, we use Quen for XYZ. And they're like, is that flexible? Can you stop doing that and use an American model instead? And I was like, oh, interesting. I totally understand where you're coming from. Yes, we can do that. But also, I'm going to talk to Harry about this tomorrow. And he's going to give me lots of wisdom.

7:25

Yeah. And he's going to tell me what to do. Did you see Poolside and Laguna? Yeah, I saw the Poolside model, and there's five open source American contenders. Let's see if I can name them all. RC, Reflection, Mistral in the West, Poolside, Thinking Machines. And then there's also Google and NVIDIA. So those are the incumbent large ones, because Google has Gemma as well. Gemma, by the way, is pretty good in terms of efficiency. If you look at Arena, you'll see that on the Pareto curves of performance versus cost, Gemma's on there. Yeah, I'm an investor in Poolside. I was actually impressed by Laguna. Yeah, great model. Yeah, it was good. I totally agree with you.

8:27

OK, totally get that. With all of these models, the question also becomes, huh, what model should I use? We spoke about Open Router earlier. And it seems like since the announcement that they were getting bored, everyone just has their own routing product. Is there value in the model routing layer? And how should I analyze that? Yeah, I absolutely think there's value in the model routing layer. That's why lots of companies are doing it. And we'll see which ones end up standing the test of time and which ones are actually a priority for the companies. I think there's an element of hype cycle right now

9:14

around routing that needs to be purged before we see who ends up actually building a great router. But routing is a very difficult technical problem. That's the first thing to realize. Because in order to route, you need to be able to take a query. And then you need to understand the nature of the query, how difficult the query is within its domain, which is hard to tell. And then you need to also understand, based on data, all the performances of the different models that are in the surf set, and also be able to quickly onboard new models that are being released, as we said,

9:52

every week. So that technical challenge, imagine if every enterprise in the world was trying to build this themselves. They wouldn't be able to do that. I'm not being rude then. How's ramp able to do it? I'm really naive here. Well, who knows how they're doing it, right? I don't know that their router is actually really deeply solving that problem. It's the Chinese. Ramp is Chinese. I heard that too. Keith Reboyd told me it was Chinese. Behind a good-looking American blonde facade. They put Eric Lyman out there and they're like, his name's Chad. Eric Lyman is a CCP member. You can send that clip to him. Yeah. Yeah. He's going to love it. He's going to love it. Yeah.

10:26

But it is so interesting. Again, we've seen so many people come out with it. Is there anything that will separate those that win from those that don't in the routing layer? And also, Nebius are coming out with their own. Fireworks have got their own. I don't know, dude, it feels pretty commoditized. Yeah. I mean, it will depend on who builds the best technology for helping people save money and get the best performance. I think all of these companies are well positioned to do it, but we'll see for whom it's a top priority and they have the machine learning team to really make it happen. I think that the other side of the debate is that, given the

11:05

complexity of the challenge, I don't think that everybody can do it. So I think the war is yet to be won. When one thinks about routing, cost is often at the center. You want to be cost and capital efficient. We thought this shit was going to get cheaper, and it hasn't got cheaper. How should we think about that? Will it just continue to not get cheaper? Will it actually get cheaper? And how should we read that? Well, I definitely think in the long run, the market will be efficient and things will get cheaper. For example, one of the things that's going to happen is that right now, Anthropic has disgustingly high

11:35

gross margins in their inference. And after they go public, the whole world is going to see that, right? We're going to see their margins because those are going to be public information, and that's going to exert downward pricing pressure on their inference. And so those types of dynamics especially... I'm not so sure. Why would that exert downward pricing pressure, just because everyone will be like, you can't have that high margins, you're price gouging? Yeah. People are going to be like, well, I know that you can do a better discount. In negotiating leverage, a standard negotiation with a private company goes like this:

11:50

I'm charging X. And then the other side says, no, it should be one third X. And they're like, I'm so sorry. I can't run a business that way. I'm just going to go home hungry. I need to make my bread too. I hope you understand. I'm not trying to price gouge you. And then the other side's like, okay, two thirds X. And then you're like, three quarters X. And they're like, make a deal. But imagine that the other side has full information about the fact that you're charging twice as much as you need to. Then it becomes easier to negotiate. Isn't that different between a good business and an average business, though? One which has pricing

12:26

power to say, listen, it's 80. And if you want to go somewhere else, by all means, but no one else does what we do. Hence Palantir and the cost plus discussion. I had CTO Pashiam on the show, and he talked to me about the cost plus being the original pricing mechanism. And now they have this. They can say, listen, sit and swivel if you want to meet in the middle, because we're the only ones who can do this. Isn't that the difference? Like Chanel. I buy Chanel for my mother. I can go to Chanel and say, I know your handbags cost 60 pounds and you're charging me 6,000. And I'll say, good, go to fucking deal then. Yeah. I mean, listen, you're right. I think Apple does this.

13:07

Apple's a great company that has such a dominant technology that they are able to charge out the nose, and their margins are probably pretty good because of it. I actually don't know Apple's margins. Do you? No idea. Yeah. Yeah. No idea. Both dumb as rocks. Yeah. We gave the disclaimer at the beginning. We can say whatever we want now. After the Eric statement, it all went downhill. Okay. So then you see that. Does Anthropic go out first? I would predict that they have all the incentives to go out first. They seem better prepared. You know, everyone likes to see free cash flow, and Anthropic is generating free cash flow.

14:02

That is massively good for the public markets. And you've seen them prepared for this. And there's been quite a bit of news about OpenAI and the internal discussions there. To what extent you believe those are true is up to you. But people are saying that they haven't been ready to IPO this year, whereas Anthropic could come as soon as October. Totally get that. And the rise of open won't impact their ability to go public this year. Well, I think that if open models really accelerate and then beat, let's say, Opus five or fable squarely across all categories, that that would be a big

14:39

business risk to them going public. But I think that they have other problems too, if that happens. Can I ask you, how significant was OpenAI and the Hugging Face gate security breach that happened a week ago? I think that was hugely significant. I think it's undervalued as a national international news incident that you're able to have a model break out of all of its safeguards and then access a bunch of company data and so on. And then in order to defend it, you need an open source model because the closed source models are refusing to do it. It's like something out of science fiction, and people didn't know that we were at that point

15:17

yet, but we absolutely are. It's just total Eliezer Yudkowsky dominance. What should we take from that then? Like Dario was right. Mythos should be curtailed, and these models business risk to them going public. But I think that they have other problems too, if that happens. Can I ask you how significant was OpenAI and the Hugging Face gate security breach that happened a week ago?

15:49

I think that was hugely significant. I think it's undervalued as a national international news incident that you're able to have a model break out of all of its safeguards and then access a bunch of company data and so on. And then in order to defend it, you need an open source model because the closed source models are refusing to do it. It's something out of science fiction, and people didn't know that we were at that point yet, but we absolutely are. It's just total Eliezer Yudkowsky dominance. What should we take from that then? Dario was right. Mythos should be curtailed, and these models have gotten too powerful too quickly. What's the subsequent takeaway from that?

16:06

My subsequent takeaway would be that we need strong external guardrails in order to make sure that these models are properly, their access controls are strong, and that they have no way of getting around them. So I think we need guardian models and also agents within our businesses. What is a guardian model?

16:16

Something that can witness the traces, that's looking over the shoulder of every agent within a business and then saying, okay, this is a safe action. This is not a safe action. Let's flag this because something weird is happening, and is equally as smart as the agent so that they're well matched. And you don't get a situation where the agent is outsmarting the guardian and able to get into trouble and mess up a business or leak all of its data. So we're going to need AI to be guarding AI because humans are going to be too slow to do that.

16:25

Well, this was my point, which is we've seen some suggestions that each model release should be approved by some form of administration. And I read this and I thought, are you freaking kidding me? No, that's not going to help. Yeah. Have you ever tried to overturn a parking ticket? Also, why should the DMV be telling me what model I can use or not? Quite funny.

16:52

It'd be totally crazy. It's like, why should we have the strongest American scientists and all of these private companies that we should incentivize to build great safeguards and maybe create some rules for them that X, Y, Z can't happen or that they're liable for huge amounts of money if corporate data gets leaked and all that stuff? I mean, it incentivizes the capitalist system to do what it does well. But the idea that we should have a central government body that tells us when it's time to release a new product versus not is crazy to me. Totally. Does that have to be a neutral, non-company, non-government body that does that regulatory role?

17:02

I think if it's not a company, it's going to be tough. I understand the need for something neutral. But you want to let the incentive system work itself out. So I would say that we should create strong safety incentives for American businesses and then regulate businesses based on the outcomes. Basically, for example, if OpenAI is letting their AI break into Hugging Face or whatever, they should get huge fines and huge scrutiny and all that stuff, as opposed to having a government process that's in charge of ensuring that this doesn't happen again, which they won't be able to do. They're not technically capable.

17:12

Do you think we're about to see a generation of cyber leaks and hacks like we've never seen before? Oh, for sure. Oh, for sure. It's going to be so insane. Can I cuss on this show? Yeah.

17:29

This is going to be so fucking insane, what happens with the cyber attacks, because here's what we see at Arena. We see another dude on the other side of the interview. They come in, they're like, hey, I want to be an infrastructure engineer at Arena, which is a great job that we're hiring for. But then the other side of it is some guy looks perfectly normal. They're passing all of our technical interviews. They're such an amazing blah, blah, blah. And then what happens at the end of it? You try to hire him, and vaporware person doesn't fucking exist. I'm not kidding. I am not kidding you. I don't know whether this is corporate espionage or cyber attacks or nation states, but people are trying to get into all of the American businesses. And we're not the only ones. This is happening everywhere. Fake people applying to companies.

17:34

I'm sorry. So you're putting out a job, people are applying, doing the tests that you set, passing them, and then when it comes to the materiality of that person being real or not, gone. Yeah. Fake person. And it's not just that we're giving them a test. They're sitting in front of people at our company. Our engineers, who are top world-class engineers, are interviewing this person and think that they're real. Why? Can you help me understand what is the benefit? They learn how you interview and hire people. I mean, the CCP are bad, but I don't think they want to steal your hiring technique. No, that's not why they do it. Why would they do it?

18:14

And I'm not saying it's the CCP. It could be anybody. It could be another company. It could be a nation state attacker. It could be a cyber hacker. Why? Because they might want access to our data or code. They might want to get double paid, like this story with this, I don't remember what that dude was. You know what I'm talking about? Yeah. That went very viral. Like a year ago. Yeah. That one kid that got four different jobs. And then he went on all the podcasts talking about it. It's another instance of that guy. And these could all be possible options, except that this person wasn't real. It was AI. Does that worry?

18:57

Yeah. Bro, it totally fucking worries me. We're going to change our whole hiring process because of this kind of stuff. So it absolutely worries me. Well, at first you need to verify that person is real. So all of our onboarding, we're considering at least making all of our onboarding in person because of this. Yeah. If you want a laptop, you got to come to the office. We got to shake your hand. We got to verify that you're real, all that kind of stuff. Absolutely. And other companies on the suit, Figma famously has done this. That's fucking wild. How hard is it to hire today in the Valley?

19:27

Oh my God. It's so crazy. It is, of course, a very, very competitive market. And the way that you see that is in terms of compensation. In order to retain fantastic people, we need to pay absolute top dollar. And we do, in order to make sure that we have the best engineers and scientists in the world. And so imagine that you're a company that's not an Arena. That's a YC company that raised a $10 million seed. It's like, fuck, man. How the hell are you supposed to hire? I think it's really tough. I think it's really tough out there. When you say top dollar, I had Brandon from McCaw on the show and he's like, oh my God. Top researchers will pay tens of millions of dollars.

19:36

Oh yeah. I'm nervous by how nonchalant you were with that. Oh yeah. If you're talking about a really top researcher, I mean, it can't be like, we're talking with somebody with many years of experience and who's really a super deep expert in their area. Many tens of thousands of citation-type researcher. And yeah, for those types of people, they're expensive. Wow. Have they all just concentrated at the frontier labs?

19:58

Many have, many have. But there's also some people who are seeing those frontier labs as big companies now. And they're saying here, I can't have a huge impact. I need to move. And so that's another demographic, actually. I think that's going to become even more extreme when the companies go public.

20:03

Can you help me? We talked about Dumb as Rocks and doing this show. I'm also an investor, for my sins, and I meet so many of these people leaving OpenAI, Anthropic, you name it. And they all kind of seem the same, if I'm totally honest. Smart people out of a great company. What will determine the Neo lab spinouts that succeed versus flame out, with a huge amount of cash going in? Yeah. I think that the Neil lab thing is really tough. So just so that we're on the same page with the audience, there's at least 75 Neil labs, and for sure, two thirds of those

20:13

companies now. And they're saying here, I can't have a huge impact. I need to move. And so that's another demographic, actually. I think that's going to become even more extreme when the companies go public. Can you help me? We talked about Dumb as Rocks and doing this show. I'm also an investor, for my sins. And I meet so many of these people leaving OpenAI, Anthropic, you name it. And they all seem the same, if I'm totally honest, smart people out of a great company. What will determine the Neo lab spin-outs that succeed versus flame out with a huge amount of cash going in?

20:21

Yeah. I think that the Neil lab thing is really tough. Just so that we're on the same page with the audience, there's at least 75 Neil labs, and for sure, two thirds of those are going to be worth nothing or they're going to be bought out for parts, right? That's going to be an acqui-hire. And so what is going to determine the winners versus the losers in that game? I think it's all about being very aggressive toward a great strategy and business model, because what's happened, and you know this better than I as an investor, is that the markets have become very P and L-driven. It's not enough just to create a model and then have a party about it. Hey, we created an AI. That is old fucking news today. It's about not just going to create a model, but do I have a sustainable business model around that? And can I generate hypergrowth in revenue? And if you're not able to do that, you're not even going to be able to raise your next round. People are raising multi-billion dollar rounds on top of just the names that are in the Neil lab with zero proof that there's any revenue-generating model behind that.

20:27

And so then the question you have to ask is, let's say I'm one of those people that's at, say, a 10 billion dollar Neil lab valuation. What do I have to believe in order to 10x my money? And the thing that you really need to believe is that if the valuation is 10 billion today, you're going to generate the revenue, let's say it's a 30x revenue multiple or 25x revenue multiple, to become a hundred billion dollar business. And so what that means is you need to be generating at least four billion dollars in revenue over the next K years, where K is something like two or three. And then if you're not doing that, everybody's going to hemorrhage out of the business. You're going to lose all your talent, and that's what we see the dynamics being.

20:33

I get you. I think there's nuance to that, candidly, which is if the company does annual tenders, you see the likes of a Mr. Al, which will be valued, I think, at 15 to 20 billion with 500 million in revenue. And so employees can take liquidity out along the way. I think 11 Labs is at 800 million in revenue, raising at 22 billion. But these companies are doing great in terms of revenue and their valuations. Those are not zero-revenue valuations. I'm talking about some valuations that are zero-revenue valuations, a 3 billion dollar company with $0 in revenue and no plan. I think Mr. is going to do great. I think 11 Labs, 11 Labs is going to be a public company, dude.

20:46

But they're not idiots doing it. So is it this amazing team from a great lab, worse comes to worst, we sell for pref stack, which is 500 million. I'm not saying whatever, whatever, but 500 million. And best case, it works, and it's a multi-hundred-billion-dollar company. I think that's a lot of the calculations. I've heard multiple people actually say this, that, hey, worst case. And that's what investors are thinking too.

20:53

Yeah, right. Investors are thinking, hey, let's say we put a couple hundred million dollars into this thing. What's the value of the team? Well, we think that just the team alone could be acquired for a billion dollars. And so the 200 million that I'm looking at is a pretty safe, zero-risk investment. Might as well put it in. But that's also the reason why the next round is the harder round. Next round's a bitch. Next round's a bitch. It sounded cooler when you said it. We got to say it at the same time. Next round's a bitch. That can be our tagline. I bet you weren't expecting this interview, huh? I don't know. Maybe. I hope you weren't. I hope you were.

21:23

No, honestly, this is so much more fun than I thought it was going to be. Okay. Cool. Can I ask you another market that I try and get my head around? Yeah, man. It's the data market. I'm an investor in McCaw. I always think it's good to put out your biases. There's so many providers at a billion dollars plus in revenue. Handshake's over a billion. McCaw's over a billion. Surge is over a billion. I might be leaving out other people, but those are the ones I know. And then hundreds of millions with the rest. What happens to this layer of the market?

21:50

Well, people are projecting growth in this market. So let's talk about why that market is a growing market and why it's hypergrowth. Mercora is obviously a generational revenue-ramp company. They've been doing great. So is Handshake. So is Surge. So is Scale. All these companies are doing great. People forget Scale. Scale is still ramping revenue well. Bro, Scale is still crushing. Still crushing, even post-fractional acqui-hire. They are. How much of that revenue is Facebook? No, I have no idea. Yeah, I don't know. A lot. Go ask Alex Wang. Agree. Okay. So why is it interesting?

22:13

So I have a thesis on hypergrowth. There's two types of hypergrowth markets that we see today. Market A is what I call scaling complements. And these are goods that are complementary goods to the scaling of AI models. And I mean that in the economic sense. A complementary good is good A and B are, the good A is a complement to good B if the demand for good B drives demand for good A. So if I have a car, gas is a complementary good to cars. The more cars are sold, the more gas is sold. And so data is one of these scaling complements. Because the bigger models scale, the more data you need. And that's a scaling law question. And so the more models you get, the bigger that they're getting, the more they're proliferating, the more businesses are training their own models, the more data you are going to need. And it's a very fundamental need.

22:21

People forget this. They think about data as a commodity. It's really not. It's actually less of a commodity than even GPUs, because in order for data to become irrelevant, humans need to become irrelevant. And that means that we've achieved AGI. And so data is a very durable need. And companies are spending on it, usually within Frontier Labs, at about 10 to 20% of the amount that they're spending on GPUs. And so if you believe in the GPU market accelerating, if you believe in the scaling of models, if you believe this is going to be a big industry that keeps accelerating and growing, then absolutely you should believe in the data market. I believe it's going to be at least $100 billion by 2030, if not a trillion.

22:21

If we expand that, okay, if we think Anthropic and OpenAI can be $3 to $5 trillion companies, let's just put that there, how big does that mean the data providers can be? McCall's reportedly raising now at 20. Does that mean that these providers will be worth $100 billion? That wouldn't be egregious, would it, to say it's 3% of the market cap of-

22:27

Yeah, I think it could easily be $100. I think these companies will easily be worth hundreds of billions of dollars. And I think they could even be worth more. The data is really the hardest part of model training. Because you need to source it, it's so dirty, nobody wants to do that shit. Nobody wants to hire all these people to generate data and then turn that into basically data plus GPUs equals model. And then the algorithms have become somewhat of a commodity because people know how to use the transformer. That's why, as you said, all the people that are coming out of the frontier labs look the same.

22:32

Can I ask you, everyone shits on these data providers for the same reason. They go, oh, but the revenue concentration is just OpenAI, Anthropic, Meta, and a couple of other providers. So is that a fair criticism? Or actually, does that not denigrate from the ultimate enterprise value of these data providers? Yeah, so I have two answers to this. The first is that I think that Silicon Valley investors have of model training. Because you need to source it, it's so dirty, nobody wants to do that shit.

22:48

Nobody wants to hire all these people to generate data and then turn that into data plus GPUs equals model. And then the algorithms have become somewhat of a commodity because people know how to use the transformer. That's why, as you said, all the people that are coming out of the frontier labs look the same. Can I ask you, everyone shits on these data providers for the same reason. They go, oh, but the revenue concentration is just OpenAI, Anthropic, Meta, and a couple of other providers. So is that a fair criticism? Or actually, does that not denigrate from the ultimate enterprise value of these data providers?

22:50

Yeah, so I have two answers to this. The first is that I think that Silicon Valley investors have become total bitches with respect to revenue concentration. It's like, what are you talking about? TSMC has revenue concentration. There are businesses that are many hundreds of billion dollar public market businesses that have revenue concentration. So I don't know what we're talking about here. There are businesses that are two-customer businesses. There are businesses that are selling to the government that have, there's one of those. And they're making huge, huge amounts of money, like an Anduril, hugely revenue-concentrated businesses. And those businesses are doing great.

22:50

Are you suggesting that venture investors have the propensity to be lazy? I would never say that. I was about to say, real. I would never say, I would never go that far. I can let you know, it's incredibly tiring sending you an email. Did you know that this competitor has just released a product? Thank you. And from Portofino. That's my one. I think that we need to have some venture investors that suck it up and put some salt on their martini glass and, okay, I'm all right with this. If you knew venture in 2026, dude, you'd know that we wear a whoop and we don't drink martinis because it impacts our sleep score. But okay. Okay. Yeah, totally.

23:32

Eight Sleep and all that stuff. Exactly. Okay. So that's one. We've become totally wusses around revenue concentration. We should bluntly embrace it.

23:53

Okay. And the second thing is that I think that a lot of data businesses are going to expand into enterprises. Of course, we plan on doing this as an evaluation business, going to enterprise and helping them with building their own AI models and all this routing stuff because we have the intelligence layer behind it that we've built on Arena. So this is obviously some place that we're going to, but many data businesses will go here as well. And the idea is that, in a world where every business needs its own AI model, why shouldn't every business need its own data? Of course they will. And the data will be part of the moat that their business accrues.

24:03

So that's on the data side. Can I ask you, when we think about, on the agent side, Anjani said I had to ask you, how does your business change as we think about the transition to full trust with agents?

24:05

Yeah. So agents is the number one priority for Arena and has been all year. People don't know this, but Arena is one of the largest consumer AI apps in the world. We're bigger than XAI. We're bigger than Hugging Face and Manus and GenSpark. It's so massive. Outside in, it's like 30 plus million monthly visitors are on Arena. And most of them are knowledge workers and prosumers, people that we call unhirable experts, people that are coming to Arena to do their real daily tasks. And in doing so, they are giving feedback that allows us to build the evaluations that we share with the world. And so it's this organic flywheel for agentic evaluations based on real data.

24:07

Why didn't you build a data business? Well, we built an evaluation business around this that allows people to understand the strengths and weaknesses of models and therefore improve them. The labs can improve their models based on the insights and data that we give them. But we also want to help businesses with this. Do you think the evaluation business is better than the data business?

24:20

I think every business in the world is going to need evaluation unambiguously. And that is the single biggest bottleneck to deploying AI, because people don't understand how to define value. All this stuff around cost per value, it's like, how do you define value? It's easy to cut costs. I can tell you to go use Gemini Flash, and that's going to be way more efficient in terms of token spend. Isn't value entirely subjective? For one, it's speed, and for others, accuracy. Do you know what I mean?

24:37

Right. Absolutely. So you can try to decompose it. I think about it as a three-pronged value proposition. There's performance, and then there's cost and latency. Cost and latency are easier to define, but performance is the tough one because the definition of performance depends on the business, depends on the use case. So at Arena, we built this pretty sophisticated pipeline for extracting organic performance measurements from agentic traces. And that's exactly where I would say that the value lies in helping businesses take advantage of their own data instead of having to purchase data in order to say which AI works best for them and even help them train their own.

24:42

What sort of revenue range are you at now? Well, so we're past a hundred million in annualized revenue run rate, and that's based on Q2 times four. And we're growing really, really fast on that front. Yep. Dick question then, how efficient are you at monetization? If you have 30 million amazing users who are unbelievably valuable in many respects, and you're only doing a hundred million, how should you- You're asking about margins. Yeah. And speed of ramp and, that good?

24:54

I mean, I think obviously we're not a free cashflow-positive business yet. We're still investing all the money that we get into making sure that we continue our rapid growth and we have a great product for all of our users and so on. But the fundamentals of the business are pretty strong. Yep. We feel great. Our investors feel great about our margins. Yeah, I'm sure they do. I would love to have been an investor. I really feel like you excluded it. I could be Greek for you for this deal. Really? Yeah. I'm a venture ambassador. We can very plastic. I once told a family- Calimera. Calimera. Hummus. Yes.

25:21

Hummus and pita. See, see, this is- We are already Greeks together, okay? I knew that this would be a productive session. Yes. Are investors over-rotating on margin also? Yeah. I don't know. I actually think margins are pretty important. We're seeing a load of businesses like your Fireworks of the world, where they're at the 30% style, mid-30s margin base. And that's very different to software margins that were 65 to 80.

25:41

Yeah. I mean, listen, profit is just margin times volume. And so you have to look at that as the calculator for the business. It's not super, super crazy. And so I don't think it's crazy to invest in these businesses. The bigger problem with businesses like that, I see these days, is that a lot of them are fundamentally GMV businesses where there's some reselling happening. I'm reselling tokens. I'm reselling GPUs and stuff like that. And those businesses are tough because, at the end of the day, you have to really think about not just the margin that you're charging in the short to medium term, but the terminal value of the good that you're providing to your customer. And so if the terminal value of the good is I'm going to host GPUs for you in order to run your models, then why should I pay you more than the cost of the electricity that it takes to run those GPUs? So the price-to-value thing is where I think you start getting into questions. That's why I think margin question is very important. And I'm not saying the margin in the short term, a seed, A, B company might not have the best margins in the world, but you should be thinking about, as this business scales and towards a public company, is it going to have a fantastic margin structure that supports a public business?

25:47

One thing that's challenging is when your customer becomes your competitor. Yeah. To what extent do you think we will see the model providers move into the application layer aggressively? We see Claude Design has actually really started to eat away at Figma. And I'm an investor in Lagora. Again, always hope people are like, oh, don't worry about Harvey. Not in any disrespect way to Harvey, the disclaimers and everything in between. Everyone at Anthropic are going to do a legal product. It's going to kill Harvey and Lagora.

26:07

Questions. That's why I think margin question is very important. And I'm not saying the margin in the short term, a Series A, B company might not have the best margins of the world, but you should be thinking about, as this business scales and towards a public company, is it going to have a fantastic margin structure that supports a public business? One thing that's challenging is when your customer becomes your competitor. Yeah.

26:26

To what extent do you think we will see the model providers move into the application layer aggressively? We see Claude Design has actually really started to eat away at Figma. And I'm an investor in Lagora. Again, I always hope people are like, oh, don't worry about Harvey. Not in any disrespect way to Harvey, the disclaimers and everything in between. Everyone at Anthropoc are going to do a legal product. It's going to kill Harvey and Lagora.

26:34

Totally. Yeah. I mean, listen, ask every business in America how they feel about this. Everybody's shaking in their boots. I have friends that are running businesses, multi-billion dollar businesses. And then what happens is that the next day, one of their biggest customers comes up and says, hey, listen, OpenAI is getting into this game. We want to work with them. We want to work with them because they're more AI-forward and you're less AI-forward because you're traditionally a SaaS business. So goodbye. Yeah, it's happening. It's absolutely happening. And I think businesses should take it really seriously. And this feeds right into this AI sovereignty debate, because a lot of what they're doing is, if I'm OpenAI and I'm Anthropic, I'm looking at who are my biggest customers, who are my customers that are winning the most in the enterprise. AI is going to commoditize, right? If inference is going to commoditize, then of course, the next best thing is for the model providers to be moving up the application layer into more and more of the application stack so that they ensure that they're not commoditized and they're getting close to the value they provided the end customer. So I absolutely think it's a risk. I think it's a risk for Ligora. I think it's a risk for Harvey. That's why Harvey's also, I mean, Harvey, the CEO of Harvey himself, is saying that his biggest competitive worry is the model labs.

26:40

I get you. But then how do you, that's a complete paradox to what we just said at the beginning about companies being scared to work with the frontier models, isn't it? No, I mean, they're scared to work with them. That's what I was saying. Is that not- They're scared to work with them and they're embracing them at the same time? Ah, you mean the customers of the- Yeah, you just said your friends running multi-billion dollar companies are like, oh, we want to work with OpenAI. I thought we just said they're scared to work with them. Yeah, it's a good question. I think you see both in the market.

27:16

Yeah. I mean, it depends on who's most automated. The thing is, okay, who's most- I think so. I think it depends actually on their GTM. If you are doing anthropic design or Claude design, dude, designers can pick up a tool and use it very efficiently. If you're Lagor or Harvey, dude, you've got to go into Cooley or Clifford Chance or any of the- Build relationships with 50-year-old white male partners who want to play golf and be told that they're great and that life is awesome. And then you've got to do deployment to junior lawyers who don't want to fucking use you because they think you're going to take their jobs too.

27:30

The deployment and the GTM is the fucking heavy lifting, and that's real world.

27:39

Totally. And there are also businesses that are less software-focused and more network-effect-focused or more operations-focused. And I think those businesses are also more likely to be adopters of the big labs. Let's say a system integrator, like an Infosys. I think more likely to be an adopter of a big lab because labs really, I think, are less likely to be competitive with an Infosys business than they are to be with some sort of scalable software product like insurance claims automation or, let's say, I think the Harvey model, like legal chatbot, let's say. I think that is tough. That's tough because I think a model lab can build that.

27:46

Do you think Salesforce will thrive in the next few years or be challenged? Salesforce themselves have a pretty strong AI strategy. So I think that those people are basically ready to go and fight in this race. So I doubt that they're going to go downhill. I think that the SaaS apocalypse has been a little bit overstated overall because people don't always understand the dynamics of those businesses and how tough it is to replicate what they've built, just also from a network perspective and a data perspective. So we'll see. We'll see.

28:01

I get you. I think if you're a ServiceNow, Salesforce, incredibly difficult, incredibly hard. I think if you're a, I love him and I interviewed him, but a Wix, less difficult, less integrated, less sticky, tougher. Do you know what I mean? I think it's all about entrenchment within enterprise. If so, golden. If not, be more nervous. Totally. Right. I'm going to do a quick-fire round with you. I'm going to say a statement. You're going to give me your immediate thoughts. Yes, sir. What have you changed your mind on in the last 12 months? Open-source model leadership. Unpack that.

28:30

Yeah. Just that I think open-source models are moving much faster than I initially thought. I think also Anthropics moving much faster than initially thought, and space moving so fast. What do you know now that you wish you'd known when you started Arena?

28:36

Man, I mean, managing people. Managing people is just the most important part of running a company. The technical stuff, I did my whole PhD on it. I spent my whole PhD proving theorems in a basement, which I loved, by the way. It was a great time. And now it's all about strategy, people, and forecasting the future, being able to look six months, a year, or two years in advance and then try to plan for that. Those are so, so important skills. Does it make sense for great, talented young people to still go to university?

28:45

It's ever more important for people to have a strong mind. And the university can be a place to develop a strong mind in terms of strong first-principles thinking, and also getting to know other people and network with them. I think that university is still a good place to go if you want to have an intellectual life, meaning where the intellectual work that you do is the primary driver of your professional career. What did you do with Arena that, with the benefit of hindsight, you wish you hadn't done?

28:57

Oh man, I had so many mistakes. I mean, at the beginning, I had no idea what I was doing. And my co-founder Jan probably knew and could see behind the corners, but I was probably too stubborn to listen to him. So, first of all, I've learned to listen to Jan more. But second, so many experiments at the beginning that I just shouldn't have wasted time with. I think the degree of focus that you need to run a company is just so extreme. You really need to do one, maybe two things extraordinarily well, and focus very, very deeply on them, pick the right ones, and focus on what's working, not on expanding into things that are not working. And that is a great lesson for me.

29:01

This is why I also agree with you, Lagor and Harvey, when it's not the main course for Anthropic to do legal, I just think you've got a really hard business when it's someone else's appetizer and it's the only thing you live and breathe. Totally. It's like priority number 12 for Anthropic is probably not high enough for Harvey and Lagor to be too scared. I'm also like, Dario, will you please just fucking solve cancer and climate change? Totally. Leave a shareholder agreement to someone else. Exactly. Honestly. You know what, though? Solving cancer is hard. It's harder than legal. A hundred percent. That's why Dario should solve it.

29:38

Well, that's why he doesn't want it, man. He just wants to take your bread. It's easier. Oh, come on, Dario. Come on. Come on, dude. Leave some bread for the rest of us. Which company will be first to $10 trillion? NVIDIA, OpenAI, or Anthropic? Hmm. I think it's hard to say not NVIDIA. I think NVIDIA is probably in the lead there. Why has NVIDIA not bounced on the rise of OpenAI? I'm an NVIDIA holder and I'm seeing flat. Why? Well, I think market probably hasn't priced it in yet. We'll see. We'll see how good these models get. Totally. Leave a shareholder agreement to someone else. Exactly. Honestly. What though? Solving cancer is hard. It's harder than legal.

30:52

A hundred percent. That's why Dario should solve it. Well, that's why he doesn't want it, man. He just wants to take your bread. It's easier. Oh, come on, Dario. Come on. Come on, dude. Leave some bread for the rest of us. Which company will be first to $10 trillion? NVIDIA, OpenAI, or Anthropic? Hmm. I think it's hard to say not NVIDIA. I think NVIDIA is probably in the lead there. Why has NVIDIA not bounced on the rise of open? I'm an NVIDIA holder, and I'm seeing flat. Why? Well, I think the market probably hasn't priced it in yet. We'll see. We'll see how good these models get.

31:54

But I think the enterprise adoption of AI is going to be another 10Xer for the industry. I think it'll 10X NVIDIA very reliably. Do you worry about the compute debt cycle and the levels of debt being taken out to fund the compute buildout? I do. I do worry about that. And I think that the reason to be worried is because if the open-source ecosystem somehow makes the cost-saving opportunity for businesses much more salient and therefore decreases the revenue of companies like OpenAI and Anthropic within the enterprise, it could lead to insolvency. I think that is the big secular trend that I would worry about if I were an investor in such markets.

32:35

My worry is we've never had such reliance on two companies to continue to hit their targets. If OpenAI and Anthropic do not continue in the strategy that they are, the music and the party goes off. And if the music goes off for everyone in the fireworks layer, no party. The rooting layer, no party. Everyone suddenly just gets the wind knocked out of them by two companies' trajectory. Totally. Yeah. I think that it's a really big deal. I think that we could use a little bit of sobering up within our industry anyway. I think that there's a lot of hype. I think that there's too much crap happening for my taste, and I'd prefer a little

33:10

bit of consolidation, actually, so we see what shakes out. I think Arena will shake out as a winner in our category, and I would love to see some of the great people that are at other businesses in our area consolidate to Arena, be able to hire them in. Where is the industry underhyped? Where is it overhyped? Well, it's interesting. I feel like everything is so hyped right now. I feel like the mechanical infrastructure for compute and data centers is relatively underhyped, like the actual cooling systems and the actual steel infrastructure. Do you know what I mean? The real physical is still underhyped. When you just, yeah,

33:29

you probably know more than me. You're in touch with the investing markets. So, I know that people are super hyped up about all of the high-bandwidth memory and the GPUs and all that stuff. That stuff is super ultra hype, right? I mean, in all stages, from public-market companies to the early stage. South Korea has called a national convening, like a community meeting, today because their stock markets are down 40%. Oh my God. A national meeting because it's down 40%? It's not a great day. Why are they down 40%? If you're a public-markets investor in South Korea, you're coming home a little bit stressed today. No, that's not good for them. Yeah.

34:07

Let's all pray for the South Koreans. Do you know what? The thing I am slightly amused by is right after everyone at SK Hynix and Samsung took home mega bonuses, then the market crashed. So why did it crash like that? What's the deal? Honestly, I think it's just a realization that everything was pretty overinflated and markets can't keep ripping for so long, I think. Interesting. There's no destabilizing factor within open or close that suggests demand is being questioned. Wow. Okay. That's why we should have a hedge fund manager on. We could do a new show hosted by Anastasios and Harry. Yeah. Yes, absolutely. Called Two Dumb Rocks. Let's do it. And

34:59

we bring exclusively us and hedge fund managers. I think it's a fucking great idea. I love it. Yeah. I actually do too. Guest one is Anjni Midha. Anjni, will you help Two Dumb Rocks? He's like, why did I fucking put this together? This is not... No, Anj would be the best guest. What's the most underrated Neolab, other than Periodic, that people aren't talking about? I don't know if I have one. I think a lot of them are overrated. I think Black Forest Labs is pretty underrated. BFL is great. Would you consider them a Neolab? Oh, don't get technical with me on semantics. Yeah. I don't know. Yeah. BFL is awesome.

35:40

Yeah, I agree. Final one for you. What are you most excited about? My mom's got MS. I'm fucking excited that chronic conditions like MS could maybe be treated. What are you excited about with the next five to 10 years? Yeah. I've always been a big proponent of AI and medicine too. I think that the level of human flourishing that's going to happen as we start to, one by one, eradicate diseases the same way that we're currently eradicating open problems in math is going to be incredible. I think it's going to be tough because the thing is that math is a closed system, and medicine, I think, will need ways of quickly iterating in a feedback loop on

36:23

biological systems. So that's the missing piece. But once we crack that, it's going to be just an extraordinary journey. It's so funny. When I interviewed Demis and I spoke about bio and medicine, it was an area where you just see his eyes light up. But it was an area where I said, hey, testing needs to change. Yeah. Fifteen years? No bueno for a lot of sufferers of chronic conditions. Yeah. And what's missing there is exactly the data layer. That's exactly one of the areas where you can clearly see that the data layer is where value is going to accrue because the GPUs are the same GPUs in both cases. The problem is that data infrastructure,

37:00

the flywheel, the data collection that you need in order to build a great biology product or a medicine product, that's tough to build. Dude, you've been a fucking epic guest. Really. I'm so grateful. It's been an amazing show, real honesty and authenticity. Most people suck as guests. You know why? Because they're not authentic, and it just comes across. You've been all... Thank you for being so great. I appreciate it. No, thank you for having me on. We'd love to do it again at some point. And you should visit the Arena office anytime that you're in the Bay Area. Yeah. Yeah.

37:44

companies on the suit figma famously has done this. That's fucking wild. How hard is it to hire today in the Valley? Oh my God. It's so crazy. It is, of course, a very, very competitive market. And the way that you see that is in terms of compensation. In order to retain fantastic people, we need to pay, absolute top dollar. And we do in order to make sure that we have the best engineers and scientists in the world. And so imagine that you're a company that's not an arena. That's like, you know, a YC company that raised a $10 million seat. It's like, fuck, man. You are, how the hell are you supposed to hire? I think it's really tough. I think it's really tough out there.

38:24

When you say top dollar, I had Brandon from McCaw on the show and he's like, Oh my God. Top researchers will pay tens of millions of dollars. Oh yeah.

38:34

I'm nervous by how nonchalant you were with that. Oh yeah. If you're talking about a really top researcher, I mean, it can't be like, we're talking with somebody with many years of experience and who's like really a super deep expert in their area. Many, you know, tens of thousands of citation type researcher. And yeah, for those types of people, they're expensive. Wow. Have they all just concentrated at the frontier labs? Many have, many have, but there's also some people who are seeing those frontier labs as big companies now. And they're saying here, I can't have a huge impact. I need to move. And so that's

39:08

another demographic actually. I think that's going to become even more extreme when the companies go public. Can you help me? We talked about Dumb as Rocks and doing this show. I'm also an investor for my sins. And I meet so many of these people leaving open AI, Anthropic, you name it. And they all kind of seem the same, if I'm totally honest, smart people out of great company. How, what will determine the Neo lab spin outs that succeed versus flame out with a huge amount of cash going in? Yeah. I think that the Neil lab thing is really tough. So just to, so that we're on the

39:43

same page with the audience, like there's at least 75 Neil labs and for sure, like two thirds of those are going to be worth nothing or like, they're going to be bought out for parts, right? That's going to be like an aqua hire. And so what is going to determine the winners versus the losers in that game? And I think it's all about being very aggressive towards a great strategy and business model, because what's happened and you know this better than I as an investor is that the markets have become very, uh, P and L driven. It's like not enough just to like create a model and then have a party

40:23

about it. Hey, we created an AI that is like old fucking news today. It's about not just going to create a model, but do I have a sustainable business model around that? And can I generate hyper growth in revenue? And if you're not able to do that, you're not even going to be able to raise your next round. People are raising multi-billion dollar rounds on top of just the names that are in the Neil lab with zero proof that there's any revenue generating model behind that. And so then the question you have to ask is let's say I'm one of those people that's at say a 10 billion

41:00

dollar Neil lab valuation. What do, what do I have to believe in order to 10 X my money? And the thing that you really need to believe is that if the valuation is 10 billion today, that you're going to generate the revenue. Let's say it's a 30 X revenue multiple or 25 X revenue multiple to become a hundred billion dollar business. And so what that means is you need to be generating at least four billion dollars in revenue over the next K years where K is something like two or three. And then if you're not doing that, everybody's going to hemorrhage out of the business. You're going to lose

41:35

all your talent, you know, and that's, that's kind of what we see the dynamics being. I get you. I think there's nuance to that candidly, which is like, if the company does annual tenders, you see the likes of a Mr. Al, which will be valued. I think it's at 15 to 20 billion with like 500 million in revenue. And so employees can take liquidity out along the way. I think 11 labs is at 800 million in revenue, raising it 22 billion. But these companies are doing great in terms of revenue and their valuations, but they didn't, those are not zero revenue valuations. I'm talking about there's some valuations that are

42:09

zero revenue valuations, $3 billion company with $0 in revenue and no plan that I mean, like I think Mr. Is going to do great. I think 11 labs, 11 labs is going to be a public company, dude. But they're not idiots doing it. So is it like, is it this amazing team from great lab, worse comes to worse. We sell for preff stack, which is 500 million. What like, I'm not saying whatever, whatever, but like 500 million and best case it works. And it's a multi hundred billion dollar company. I think that's a lot of the calculations. I've heard multiple people actually say this is that, Hey, you know, worst case. And that's what investors are thinking too.

42:47

Yeah. Right. Investors are thinking like, Hey, let's say we put a couple hundred million dollars into this thing. What's the value of the team? Well, we think that the, just the team loan could be acquired for a billion dollars. And so the 200 million that I'm looking at is like pretty safe, zero risk investment might as well put it in. But that's also the reason why the next round is the harder round. Next round's a bitch. Next round's a bitch. It sounded cooler when you said it.

43:20

We got to say it at the same time. Next round's a bitch. That'll be like our, that can be our tagline. I bet you weren't expecting this interview, huh? I don't know. Maybe I hope you weren't. I hope you were. No, honestly, this is so much more fun than I thought it was going to be. Okay. Get that. Cool. Can I ask you another market that I try and get my head around? Yeah, man. It's the data market. I'm an investor in McCaw. I always think it's like good to put out your biases. There's so many providers at a billion dollars plus in revenue. Handshake's over a billion. McCaw's over a billion. Surge is over a billion. I might be leaving out other people,

44:00

but those are the ones I know. And then hundreds of millions with the rest. What happens to this layer of the market? Well, people are projecting growth in this market. So let's talk about why that market is a growing market and why it's hyper growth. I mean, Mercora obviously is a generational revenue ramp company. They've been doing great. So is Handshake. So is Surge. So is Scale. All these companies doing great. People forget Scale. Scale is still ramping revenue well. Bro, Scale is still crushing. Still crushing even post fractional aqua hire. They are. How much of that revenue is Facebook? No, I have no idea. Yeah. I don't know. A lot. Go ask Alex Wang. Agree.

44:42

Okay. So why is it interesting? So I have a thesis on hyper growth. There's two types of hyper growth markets that we see today. Market A is what I call scaling complements. And these are goods that are complementary goods to the scaling of AI models. And I mean that in the economic sense, a complementary good is good A and B are, the good A is a complement to good B if the demand for good B drives demand for good A. So if I have a car, gas is a complementary good to cars. The more cars are sold, the more gas is sold. And so data is one of these scaling complements. Because the bigger models scale,

45:24

the more data you need. And that's a scaling law question. And so the more models you get, the bigger that they're getting, the more they're proliferating, the more businesses are training their own models, the more data you are going to need. And it's a very fundamental need. People forget this. They think about data as a commodity. It's really not. It's actually less so of a commodity than even GPUs. Because in order for data to become irrelevant, humans need to become irrelevant. And that means that we've achieved AGI. And so data is a very durable need. And companies are spending on it, usually within Frontier Labs, at about 10 to 20%

46:04

about the amount that they're spending on GPUs. And so if you believe in the GPU market accelerating, if you believe in the scaling of models, if you believe this is going to be a big industry that keeps accelerating and growing, then absolutely you should believe in the data market. I believe it's going to be at least $100 billion by 2030, if not a trillion. If we expand that, okay, if we think Anthropoc and OpenAI can be $3 to $5 trillion companies, let's just put that there. How big does that mean the data providers can be? Like McCall's reportedly raising now at 20. Does that mean that these providers will be worth $100 billion? That wouldn't

46:36

be egregious, would it, to say it's 3% of the market cap of- Yeah, I think it could easily be $100. I think these companies will easily be worth hundreds of billions of dollars. And I think they could even be worth more. The data is really the hardest part of model training. Because you need to source it, it's so dirty, nobody wants to do that shit. Nobody wants to hire all these people to generate data. And then, you know, turn that into basically, data plus GPUs equals model. And then the algorithms have become somewhat of a commodity because people know how to use the transformer. That's why, as you said, all the people that are

47:16

coming out of the frontier labs look the same. Can I ask you, everyone shits on these data providers for the same reason. They go, oh, but the revenue concentration is just OpenAI, Anthropic, Meta, and a couple of other providers. So is that a fair criticism? Or actually, does that not denigrate from the ultimate enterprise value of these data providers? Yeah, so I have two answers to this. The first is that I think that Silicon Valley investors have become total bitches with respect to revenue concentration. It's like, what are you talking about? Like TSMC has revenue concentration. There's businesses that are like many hundreds of

47:56

billion dollar public market businesses that have revenue concentration. So I don't know what we're talking about here. There's businesses that are like two customer businesses that there's businesses that are selling to the government that have, there's like one of those. And they're making like huge, huge amounts of money, like an Anduril, hugely revenue concentrated businesses. And those businesses are doing great. Are you suggesting that venture investors have the propensity to be lazy? I would never say that. I was about to say real. I would never say, I would never go that far. I can let you know, it's incredibly tiring sending you an email.

48:36

Did you know that this competitor has just released a product? Thank you. And from Portofino. That's my one is I think that we need to like have some venture investors that like kind of suck it up and like put some salt on their martini glass and like, okay, I'm all right with this. If you knew venture in 2026, dude, you'd know that we wear a whoop and we don't drink martinis because it impacts our sleep score. But okay. Okay. Yeah, totally. Eight sleep and all that stuff. Exactly. Okay. So that's one. We've become totally wusses around revenue concentration. We should bluntly embrace it.

49:10

Okay. And the second thing is that I think that a lot of data businesses are going to expand into enterprises. Of course, you know, we plan on doing this as an evaluation business is going to enterprise and helping them with building their own AI models and all this routing stuff because we have the intelligence layer behind it that we've built on Arena. So this is obviously some place that we're going to, but many data businesses will go here as well. And the idea is that in a world where every business needs its own AI model, why shouldn't every business need its own data? Of course they will. And the data will be part of the moat that their business accrues.

49:51

So that's on the data side. Can I ask you, when we think about like on the agent side, Anjani said I had to ask you, how does your business change as we think about the transition to full trust with agents? Yeah. So agents is the number one priority for Arena and has been all year. People don't know this, but Arena is one of the largest consumer AI apps in the world. We're bigger than like XAI. We're bigger than like Hugging Face and Manus and GenSpark where it's so massive. Like if you, like outside in, it's like 30 plus million monthly visitors are on Arena. It's, it's, and because,

50:32

and most of them are knowledge workers and prosumers, people that we call unhirable experts, people that are coming to Arena to do their real daily tasks. And in doing so, they are giving feedback that allows us to build the evaluations that we share with the world. And so it's this organic flywheel for agentic evaluations based on real data. Why didn't you build a data business? Well, we built an evaluation business around this that allows people to understand the strengths and weaknesses of models and therefore improve them. The labs can improve their models based on, you know, the insights and data that we give them. But we also want to help businesses with this.

51:08

Do you think the evaluation business is better than the data business? I think every business in the world is going to need evaluation unambiguously. And that is the single biggest bottleneck to deploying AI, because people don't understand how to define value. All this, like stuff around cost per value. It's like, how do you define value? It's easy to cut costs. I can tell you to go use, you know, Gemini flash. And that's going to be like way more efficient in terms of token spend. Isn't value entirely subjective? Like for one, it's speed and for others accuracy for what, do you know what I mean?

51:43

Right. Absolutely. So you can try to decompose it. I think about it as three, a three, three pronged value proposition. There's performance. And then there's costs and latency. Costs and latency are easier to define, but performance is the tough one because the definition of performance depends on the business, depends on the use case. So at Arena, we built this pretty sophisticated pipeline for extracting organic performance measurements from agentic traces. And that's exactly where I would say that the value lies in helping businesses take advantage of their own data instead of having to purchase data in order to say which AI works best for them

52:27

and even help them train their own. What sort of revenue range are you at now? Well, so we're past a hundred million in annualized revenue run rate, and that's based on like Q2 times four. And we're growing really, really fast on that front. Yep. Dick question then, how efficient are you at monetization? If you have 30 million amazing users who are unbelievably valuable in many respects, and you're only doing a hundred million, how should you- You're asking about margins. Yeah. And like speed of ramp and like, that good? I mean, like, I think obviously we're not like a free cashflow positive business yet. We're still

53:06

investing all the money that we get into making sure that we continue our rapid growth and we have a great product for all of our users and so on. But the fundamentals of the business are pretty strong. Yep. We feel great. Our investors feel great about our margins. Yeah, I'm sure they do. I would love to have been an investor. I really feel like you exclude it. You know, I could be Greek for you for this deal. Really? Yeah. I can, I'm a venture ambassador. We can very plastic. I once told a family- Calimera. Calimera. Hummus. Yes. Hummus and pita. See, see, this is- We are already Greeks together, okay?

53:42

I knew that this would be a productive session. Yes. Are investors over-rotating on margin also? Yeah. I don't know. I actually think margins are pretty important. We're seeing a load of businesses like your fireworks of the world, where they're at the 30% style, mid-30s margin base. And that's very different to software margins that were 65 to 80. Yeah. I mean, listen, profit is just like margin times volume. And so you have to look at that as the calculator for the business. It's not like super, super crazy. And so I don't think it's crazy to invest in these businesses. The bigger problem with businesses like that I see these days

54:24

is that a lot of them are fundamentally GMV businesses where there's like some reselling happening. I'm reselling tokens. I'm reselling GPUs and stuff like that. And those businesses are tough because at the end of the day, you have to really think about not just the margin that you're charging in the sort of short to medium term, but the terminal value of the good that you're providing to your customer. And so if the terminal value of the good is I'm going to host GPUs for you in order to run your models, then why should I pay you more than like the cost of the electricity that it takes

55:03

to run those GPUs? So the sort of like price to value thing is where I think you start getting into questions. That's why I think margin question is very important. And I'm not saying the margin in the short term, a series, you know, seed A, B company might not have the best margins of the world, but you should be thinking about as this business scales and towards a public company, is it going to have a fantastic margin structure that supports, you know, a public business? One thing that's challenging is when your customer becomes your competitor. Yeah. To what extent do you think we will see the model providers move into the application layer

55:40

aggressively? We see Claude Design has actually really started to eat away at Figma. And I'm an investor in Lagora. Again, always hope people are like, oh, don't worry about Harvey. Not in any disrespect way to Harvey, the disclaimers and everything in between. Everyone's at Anthropoc are going to do a legal product. It's going to kill Harvey and Lagora. Totally. Yeah. I mean, listen, ask every business in America how they feel about this. Everybody's shaking in their boots. I have friends that are running businesses, multi-billion dollar businesses. And then what happens is that the next day, one of their biggest customers comes up and says,

56:14

hey, listen, OpenAI is getting into this game. We want to work with them. We want to work with them because they're more AI forward and you're less AI forward because you're, you know, traditionally a SaaS business. So goodbye. Yeah, it's happening. It's absolutely happening. And I think businesses should take it really seriously. And this feeds right into this AI sovereignty sort of debate, because a lot of what they're doing is, you know, if I'm open and I'm, and I'm Anthropic, I'm looking at who are my biggest customers, who are my customers that are winning the most in the

56:45

enterprise. AI is going to commoditize, right? If like inference is going to commoditize, then of course, the next best thing is for the model providers to be moving up the application layer in order to more and more of the application stack so that they ensure that they're not commoditized and they're getting close to the value they provided the end customer. So I absolutely think it's a risk. I think it's a risk for Ligora. I think it's a risk for Harvey. That's why Harvey's also, I mean, Harvey, the CEO of Harvey himself is saying that, you know, his biggest competitive worry is the model labs. I get you. But then how do you, that's a complete paradox to what we just

57:22

said at the beginning about companies being scared to work with the frontier models, isn't it? No, I mean, they're scared to work with them. That's what I was saying. Is that not- They're scared to work with them and they're embracing them at the same time? Ah, you mean the customers of the- Yeah, you just said your friends running multi-billion dollar companies are like, oh, we want to work with OpenAI. I thought we just said they're scared to work with them. Yeah, it's a good question. I think you see both in the market. Yeah. I mean, it depends on who's most automated. The thing is that like, okay, who's most-

57:51

I think so. I think it depends actually on their GTM. If you are doing anthropic design or Claude design, dude, designers can pick up a tool and use it very efficiently. If you're Lagor or Harvey, dude, you've got to go into Cooley or Clifford Chance or any of the- Build relationships with 50-year-old white male partners who want to play golf and be told that they're great and that life is awesome. And then you've got to do deployment to junior lawyers who don't want to fucking use you because they think you're going to take their jobs too. The deployment in the GTM is the fucking heavy lifting and that's real world.

58:28

Totally. And there's also businesses that are less software-focused and more network effect-focused or more operations-focused. And I think those businesses are also more likely to be adopters of the big labs. Let's say system integrator, like an Infosys. I think more likely to be an adopter of a big lab because labs really, I think, less likely to be competitive with an Infosys business than they are to be with some sort of a scalable software product like insurance claims automation or let's say, I think the Harvey model, like legal chatbot, let's say. I think that is tough. That's tough because I think a model lab can build that. Do you think Salesforce will thrive

59:16

in the next few years or be challenged? You know, Salesforce themselves have a pretty strong AI strategy. So I think that those people are basically like ready to go and fight in this race. So I doubt that they're going to like go downhill. I think that the SaaS apocalypse has been a little bit overstated overall because people don't understand always the dynamics of those businesses and how tough it is to replicate what they've built. Just also from a network perspective and a data perspective. So we'll see. We'll see. I get you. I think if you're a service now, Salesforce, incredibly difficult, incredibly hard. I think if you're a, I love him and I interviewed him,

1:00:04

but like a Wix, less difficult, less integrated, less sticky, tougher. Do you know what I mean? I think it's all about entrenchment within enterprise. If so, golden. If not, be more nervous. Totally. Right. I'm going to do a quick fire round with you. I'm going to say a statement. You're going to give me your immediate thoughts. Yes, sir. What have you changed your mind on in the last 12 months? Open source model leadership. Unpack that. Yeah. Just that I think open source models are moving much faster than I initially thought. I think also Anthropics moving much faster than initially thought and space moving so fast. What do you know now that you wish you'd

1:00:42

known when you started Arena? Man, I mean, managing people. Managing people is just the most important part of running a company. The technical stuff, you know, I did my whole PhD on it. I spent like my whole PhD proving theorems in a basement, which I loved, by the way. It was like a great time. And now it's all about strategy people and forecasting the future, being able to like look six months, a year or two years in advance and then try to plan for that. Those are so, so important skills. Does it make sense for great talented young people to still go to university? It's ever more important for people to have a strong mind. And the university can be a place

1:01:18

to develop a strong mind in terms of strong first principles thinking, and also getting to know other people and network with them. I think that university is still a good place to go if you want to have an intellectual life, meaning where the work, the intellectual work that you do is the primary driver of your professional career. What did you do with Arena that with the benefit of hindsight you wish you hadn't done? Oh man, I had so many mistakes. I mean, at the beginning, I had no idea what I was doing. And I, you know, my co-founder Jan probably knew and could see behind the corners, but I was probably too stubborn to listen to him. So,

1:01:54

first of all, I've learned to listen to Jan more. But second, it's, you know, so many like experiments at the beginning that I just shouldn't have wasted time with. I think the degree of focus that you need to run a company is just so extreme. You really need to do one, maybe two things extraordinarily well, and focus very, very deeply on them, pick the right ones and focus on what's working, not on expanding into things that are not working. And that is a, that is a great lesson for me. This is why I also, I agree with you, Lagor and Harvey, like when it's not the main course for

1:02:28

Anthropic to do legal, I just think you've got a really hard business when it's someone else's like appetizer and it's the only thing you live and breathe. Totally. It's like priority number 12 for Anthropic is probably not high enough for Harvey and Lagor to, to be too scared. I'm also like, Dario, will you please just fucking solve cancer and like climate change? Totally. Leave a shareholder agreement to someone else. Exactly. Like honestly. You know what though? Solving cancer is hard. It's harder than legal. A hundred percent. That's why Dario should solve it. Well, that's why he doesn't want it, man. He just wants to take your bread. It's easier. Oh, come on, Dario.

1:03:08

Come on. Come on, dude. Leave some bread for the rest of us. Which company will be first to $10 trillion? NVIDIA, OpenAI or Anthropic? Hmm. I think it's hard to say not NVIDIA. I think NVIDIA is probably in the lead there. Why have NVIDIA not bounced on the rise of open? I'm an NVIDIA holder and I'm seeing flat. Why? Well, I think market probably hasn't priced it in yet. We'll see. We'll see how good these models get. But I think the enterprise adoption of AI is going to be another 10Xer for the industry. I think it'll 10X NVIDIA very reliably. Do you worry about the compute debt cycle and the levels of debt being taken out

1:03:48

to fund the compute build out? I do. I do worry about that. And I think that the reason to be worried is because if the open source ecosystem somehow makes the cost saving opportunity for businesses much more salient and therefore decreases the revenue of companies like OpenAI and Anthropic within the enterprise, that it could lead to insolvency. I think that is the big secular trend that I would worry about if I were an investor in such markets. My worry is we've never had such reliance on two companies to continue to hit their targets. If OpenAI and Anthropic do not continue in the strategy that they are, the music and the party

1:04:31

goes off. And if the music goes off for everyone in the fireworks layer, no party. The rooting layer, no party. Everyone suddenly just gets the wind knocked out of them by two companies trajectory. Totally. Yeah. I think that it's a really big deal. You know, I think that we could use a little bit of um, sobering up within our industry anyway. Uh, I think that there's a lot of hype. I think that there's a lot of, um, there's, there's too much crap happening for my taste and I prefer a little bit of consolidation actually. So we see what shakes out. I think arena will shake out as a winner in our

1:05:08

category. Um, and I would love to see some of the great people that are at other businesses in our area, uh, consolidate to arena, be able to hire them in. Where is the industry under hyped? Where is it over hyped? Well, it's interesting. I mean, I feel like everything is so hyped right now. I feel like the, the, the mechanical infrastructure for compute and data centers is relatively under hyped, like the actual cooling systems and the actual like steel, steel infrastructure. Do you know what I mean? Like the real physical is still under hyped. When you just, yeah, you probably know more than me. You're in touch with the investing markets. So, I mean, like, I know

1:05:48

that people are super hyped up about all of the high bandwidth memory and the GPUs and all that stuff. That stuff is super ultra hype. Right. I mean, for, on basically in all stages from public market companies to the early stage, South Korea have caused, called a national convene, like community meeting today because their stock markets are down 40%. Oh my God. A national meeting because it's not 40%. It's not a great day. Why are they down 40%? If you're a public markets investor in, in South Korea, you're, you're, you're coming home a little bit stressed today. No, that's not good for them. Yeah.

1:06:26

Let's all pray for the, let's pray for the South Koreans. Do you know what I, the thing I am slightly amused by is right after everyone at SK Hynix and Samsung took home like mega bonuses, then the market crashed. So why did it crash like that? What's the deal? Honestly, I think it's just a realization that, you know, everything was pretty overinflated and markets can't keep ripping for so long. I think. Interesting. There's no, there's no destabilizing factor within open or close that suggests demand is being questioned. So. Wow. Okay. That's why we should have a hedge fund manager on. We could do a

1:07:02

new show hosted by Anastasios and Harry. Yeah. Yes, absolutely. Called Two Dumb Rocks. Let's do it. And we bring exclusively us and hedge fund managers. I think it's a fucking great idea. I love it. Yeah. I actually, I actually do too. Guest one is Anjni Midha. Anjni, will you help Two Dumb Rocks? He's like, why did, why did I fucking put this together? This is not. No, Anj would be the best guess. What's the most underrated Neolab other than Periodic that people aren't talking about? Uh, I don't know if I have one. I think a lot of them are overrated.

1:07:38

I think Black Forest Labs is pretty underrated. BFL is great. Would you consider them a Neolab? Oh, don't get technical with me on semantics. Yeah. I don't know. Yeah. BFL is awesome. Yeah. I agree. Um, final one for you. What are you most excited about? My mom's got MS. I'm fucking excited that chronic conditions like MS could maybe be treated. What are you excited about with the next five to 10 years? Yeah. I mean, I've always been a big proponent of AI and medicine too. I think that the like level of just human flourishing that's going to happen as we start to one by one eradicate diseases the same way that we're currently eradicating open problems in math

1:08:18

is going to be incredible. I think it's going to be tough because the thing is that math is a closed system and medicine. I think you'll need to figure out ways of quickly iterating in a feedback loop on biological systems. So that's the missing piece. But once we crack that, it's going to be just an extraordinary journey. It's so funny when I interviewed Demis and I spoke about like bio and medicine, it was, it was an area where you just see his eyes light up. Um, but it was an area where I said, Hey, testing needs to change. Yeah. 15 years. No bueno for a lot of sufferers of chronic conditions.

1:08:53

Yeah. And you know, what's missing that is exactly the data layer. That's exactly one of the areas where the data layer, where you can clearly see that the data layer is where value is going to accrue because the GPUs are the same GPUs in both cases. The problem is that that data infrastructure, the flywheel, the data collection that you need in order to build a great biology product or a medicine product. That's tough to build. Dude, you've been a fucking epic guest. Like really? Like I'm so grateful. Um, it's been an amazing show, real honesty and authenticity. Most people suck

1:09:28

as gas, you know why? Cause they're not authentic and it just comes across. Uh, you've been all, thank you for being so great. I appreciate it. No, thank you for having me on. We'd love to do it again at some point. And you should visit the arena office anytime that you're in, uh, the Bay area. Yeah. Yeah.

Reading tools

Type to find a passage

Appearance
Ask this transcript

Notebook

Export Markdown

Add a note