Why are so many people so sure that the big AI providers are losing money on inference? It reminds me of the comments about how Uber can never make money. Their unit economics were fine and they were only losing money because they chose to do so on customer acquisition.
In this case, I don't think it's even obvious that inference should be money losing today. API cost for frontier models has gone way up and the people claiming some cost for inference are relying on made up assumptions.
@danluu my impression from friends at frontier labs is that inference isn't anywhere close to losing money right now. Training is a huge money pit of course but inference is pretty profitable just measured on a cost per token basis, even at the subsidised monthly subscription rates.
(Obviously this is hearsay and you shouldn't treat it as particularly credible, but I personally think it's very likely to be true)
@DRMacIver It seems like inference could be massively profitable. Whether or not training costs offset is a tuning knob like Uber had.
Given how much OpenAI and Anthropic have jacked up prices recently (if you want their latest models), and how there's no real competitor (I try Gemini every once in a while and it's never been in the same league for coding, though it is better at some things), it doesn't seem like OAI and Antropic are competing on price, which leaves room for a lot of profit.
If you read widely cited folks saying AI economics are bad today (e.g., Ed Zitron from the quote), they're not credible. Sure, Ed writes rants, swears a lot, and throws the word "scam" around, all of which gets a lot of clicks, but the reasoning seems wrong.
If I were going to be skeptical about the economics of AI, it would be some kind of reasoning about how cheap models keep improving, so the frontier labs need to stay ahead (I don't follow it closely enough to know if that's plausible).
@danluu the pro models are drastically better than the free models, so this is important
Question is for how much longer will this be the case. Is there a ceiling for LLMs? Or are the limiting factors the size of a consumer hardware GPU vs the much larger server farm GPU?
@danluu I'm skeptical about the economics of AI, simply because it looks too much like the economics of cryptocurrency. It's simply an investment to be bought low and sold high. No one cares about selling a product or having a business model to any extent except to woo investors. Making a valuable product or being able to turn a profit is the problem of whoever holds the bag down the line. Everything about how AI is marketed, the tone and formatting of e.g. chatgpt output, and the "impending AGI doomsday" stunt of 2025, looks like it is carefully crafted to sell company stock rather than product.
@danluu i don't think the guy is a messiah or anything, but beyond the swearing, Ed Zitron's analysis is a hell of a lot more credible than saying it's not credible with the only counterargument being that it sounds wrong. what about is sounds wrong? why is he not credible? why are those that say that these companies who have not demonstrated returns and are racing to ratchet up cash in while limiting usage on the subscription models (one of his main criticism being that subscription models make no economic sense because they decouple the driver of cost from the revenue mechanism) are profitable?
@danluu part of the argument is that not just that it might not be profitable now, but that the amount of profitable that it would need to be to justify the amount of capital expenditure that has already been made and is promised is numerically impossible. JPMorgan estimated 1.2 trillion in AI debt back in december 2025, goldman sachs estimates another 500 billion in 2026. Where is the evidence that inference is profitable enough to pay off 1.7 trillion? If it was really profitable, all the publicly traded AI companies would be screaming this at the top of their quarterly reports.
@jonny @danluu I do agree that Ed is kind of a prick, can be fixated on dubious claims and extrapolates a lot, but overall his reporting is the one with the most precise data. Maybe inference is or will be profitable? We have no reliable sources on either side of the claim. The leading companies would probably shout it everywhere if it was, and they usually lie quite a bit to embellish the picture so the fact that they do not really talk about it is telling.
Maybe the industry can be sustained by every company deciding to pay $1000 seats apiece per engineer in perpetuity, but that sounds like a bad idea in aggregate.
@danluu the pro models are drastically better than the free models, so this is important
@danluu I think it’s people hoping that the price increases are just a lower subsidy and not at cost or profitable. I’m not sure the evidence available necessarily disproves that either.
Install bonfire.cafe
Get the full app experience
