Foundation Models vs App Layer: Who Actually Captures the AI Value?
Foundation model companies who are playing in the app layer will win
Introduction
Over the past few months, VCs have debated where value accrues in the AI stack. Is it the foundation model companies like OpenAI and Anthropic? Or is it the app layer companies like Harvey, Glean, and Cursor? In this essay, I argue that the trillion-dollar outcomes will still accrue at the app layer—but the biggest winners at this layer will be foundation model companies moving upstack, rather than new app layer startups.
Yes — I stand by my previous prediction (in which I shared on X) that value will accrue at the app layer. However, the winners will be foundation model companies moving upstack, not new startups building in the space. There are certain caveats that I’ll outline later in the essay, but foundation model companies are the biggest beneficiaries of general horizontal solutions in the app layer.
Conventional Wisdom
Conventional wisdom tells us the app layer is where you get truly big outcomes. These businesses are easy to understand, have simple business models, and generally have extremely large TAMs.
During the DeepSeek frenzy, investors doubled down on the app layer, believing that costs would come down for app layer companies. By extension, costs should come down for consumers as well. And this is true for the most part! AI has experienced the steepest cost depreciation of any recent technology, with token costs falling by 99% over the past 18 months.
So, what’s the issue? This isn’t translating too well on the income statements just yet. For app layer companies, especially coding copilots, there’s very little differentiation between company A and company B (you might be able to guess the companies I’m referring to).
Because most app-layer companies rely heavily on similar underlying models (GPT), their products quickly commoditize. As competition grows, they’re forced into price wars that erode margins despite falling token costs.
That said, it’s difficult for a GPT wrapper to become a billion-dollar business. But, this doesn’t mean it’s impossible to build a business with great financial outcomes. Companies like CalAI are hitting $30m in ARR, a seemingly simple GPT wrapper.
Hidden Advantages of Foundation Models
Why do I believe foundation models are going to be the winners? My contention falls broadly under three categories:
Vertical mobility
Cost efficiency
Ecosystem lock-in
Vertical Mobility
It’s easy for foundation models to move upstack, but difficult for app layer companies to move downstack. (All the foundation model companies are already moving upstack, I’ll provide some examples in the next section).
App layer companies struggle to move downstack because of capital, data, and talent constraints. Foundation models, especially leading incumbents, have everything they need to move upstack. More importantly, they have distribution, which is the key to winning in this day and age. ChatGPT, for example, has more monthly users than Wikipedia!
Sam Altman said that the opportunity lies in the app layer, making OpenAI’s strategic direction abundantly clear. This explains the motivations for moving upstack, but why does an app layer OpenAI beat out an app layer using OpenAI and Anthropic’s APIs?
Cost efficiency
This is where cost efficiency kicks in. Because OpenAI avoids API pricing markups, it can offer similar or better products at structurally lower costs, undercutting API-reliant competitors.
OpenAI and Anthropic need to pass through the cost to businesses that use their APIs; as a result, there’s always an upcharge when a business like Cursor or Harvey calls an API. According to o3, OpenAI and Anthropic price their APIs anywhere from ~2× to >10× above raw GPU cost. This “upcharge” goes away when they’re building the product in-house.
This is important when looking at gross margins and free cash flows, especially for a business that’s beginning to mature. As mentioned earlier, generic businesses that are highly elastic are doomed especially if switch costs are low and they’re competing on price. In fact, as a general rule of thumb, it’s always a bad idea to make your competitive advantage based on costs.
Ecosystem lock-in
We’ve established that it’s easy for foundation model companies to move upstack, and it’s difficult for the app layer companies to move downstack. We’ve also established that foundation model companies are destined to have improved margins compared to app layer companies. So, what is ecosystem lock-in?
To put it simply, this is customer captivity and distribution. People know, use, and love ChatGPT. Most consumers already pay for ChatGPT subscriptions. Why buy a new subscription elsewhere when ChatGPT conveniently bundles similar services like Codex or Deep Research?
OpenAI continues bundling additional tools like Deep Research and Operator into their subscription plans, creating deep ecosystem lock-in. As user dependency grows, OpenAI gains substantial pricing power without risking significant churn.
This is the power of distribution.
OpenAI Case Study
Let’s talk about our favorite company, OpenAI. They’ve made it abundantly clear that they’re moving up to the app layer. Last week, OpenAI released Codex, a coding agent competing against the likes of Cursor. Just a few weeks back, Sam Altman tweeted that he was hiring Fidji, the CEO of Instacart, to become the CEO of Applications at OpenAI. If this isn’t clear what Sam’s intentions are, I’m not sure what is.
OpenAI is also actively hiring more FDE/solution engineer roles, demonstrating their commitment to move to serving enterprise clients. OpenAI is most likely looking to break into the enterprise search space, competing directly with companies like Glean.
Looking at Anthropic, we see a similar pattern. Their partnership with Amazon demonstrates their intent. On the consumer front, they’ve been pushing out influencer marketing and content on social media, and recently launched a campus ambassador program. Again, these multi-billion-dollar companies understand the alpha on the app layer, and they’re not letting these startups win.
How Can App Layer Companies Thrive?
As I tweeted, app layer companies can survive if they check off these three boxes:
Build in an extremely vertical niche that a horizontal/generic solution can’t beat (think vertical SaaS solutions like Veeva or ServiceTitan)
Own proprietary data that foundation models don’t have access to (e.g. Harvey acquiring vLex would be a huge win)
Strong data feedback loop (you should continuously improve your product based on the data and feedback that is being collected by users using the product)
Unfortunately, these vertical companies will be inherently smaller in scale compared to horizontal solutions due to a smaller TAM (even when accounting for AI’s TAM expansion via labor). However, this doesn’t mean billion-dollar outcomes aren’t possible!
Ultimately, we must avoid chasing trendy app-layer startups and instead deeply evaluate what genuinely makes these companies defensible.