Falling AI inference cost is a trap if you built for scarcity

‍The model got cheap. The hard part just moved.

AI inference cost per token is collapsing, and most teams are reading that as good news. It is a trap. Per-token prices are falling while total AI bills explode, because the moment intelligence gets cheap, teams spend far more of it, and almost nobody is managing how. The scarce resource is no longer the model. It is the judgment to run many cheap agents at once and stay accountable for what they do.

‍Every serious AI product built in the last three years has been shaped by one quiet assumption: model calls are expensive, so use as few as you can. Pick a capable model, feed it good context, get your answer, move on. That instinct was correct. It is also about to be wrong, and the gap between those two facts is where the next round of winners and losers gets decided.

On August 13, Google shipped Gemini 3.7 Flash, a model aimed squarely at coding and agent workloads, at an introductory 75 cents per million input tokens and $3.75 per million output. It landed 23 days after the model it replaced. Read that cadence again. A materially better agent model, at half the blended price of its predecessor, three weeks later. This is not a product launch. It is a trend line, and it points down fast enough that any architecture you freeze today is depreciating by the time it ships.

There is a detail in the pricing everyone is glossing over. That 75-cent rate is a discount that expires December 31, and doubles on January 1. So the lesson is not simply “intelligence is getting cheaper.” It is that the price of intelligence is now a moving target you do not control, which is a very different thing to build a company on.

And here is the twist that should stop anyone celebrating cheaper tokens. AI inference cost per token is falling fast, yet enterprise bills are climbing at the same time. The reason states in one line: when the price per token drops, teams run so many more tokens that total spend rises anyway, so a lower price per call routinely produces a higher bill. Uber burned through its entire 2026 AI coding budget in four months, and Microsoft canceled most of its internal Claude Code licenses over cost. The per-unit price dropped and the total still exploded, because the moment intelligence got cheap, engineers spent dramatically more of it. Cheaper inputs do not produce cheaper systems on their own. They produce more consumption, and whether it is worth anything depends entirely on whether someone is managing it.

I have seen this movie. The cheap thing stops being the thing you ration.

Here is the shift most teams are underpricing, stated plainly. When a resource gets cheap enough, the winning move flips from conserving it to spending it deliberately. The teams that keep optimizing for thrift lose to the teams that learn to spend.

I have watched a smaller version of this reshape my own business as bandwidth and cloud compute fell. The habits you build under scarcity become the thing holding you back the moment scarcity lifts, and almost nobody notices the switch in time.

This is not a prediction. It is a rerun.

Carver Mead, the Caltech physicist who coined the term Moore's Law, spent decades watching transistors fall from a dollar apiece toward nothing. His framing of what happened next is the most useful sentence in computing history for understanding this moment. What happens when things get nearly free? You waste them. You shift out of conservation mode and into exploitation mode. Alan Kay “wasted” absurd numbers of newly cheap transistors to build the graphical user interface, which looked profligate right up until it became the way everyone computes.

The people who won that era were not the ones who kept hand-optimizing to save transistors. They were the ones who understood the price was collapsing and started spending the surplus on things that were previously unthinkable. Inference is on the same curve now, and the same reflex is about to become the same liability.

Redundancy becomes the point, not the waste

Traditional engineering treats redundant computation as waste to be trimmed. With probabilistic systems, redundancy is often the thing that makes the output trustworthy, and that changes what a sane architecture looks like.

Consider a consequential task, say an agent preparing a financial analysis. The scarcity-era design sends it to your best model once and hopes. The abundance-era design runs several cheap agents at the problem independently, has another evaluate their outputs against each other, retries automatically when confidence is low, and escalates only the genuinely hard sub-problem to an expensive frontier model. That second design burns far more tokens. It is also far more likely to be right, and as the price per token keeps falling, the math that once made it indefensible starts making it obvious.

Look at what this does to the shape of the system. AI agent architecture is quietly moving from “one model, one answer” to “many attempts, verification, then action.” Model selection stops being a setup decision and becomes a runtime one: cheap workhorse for classification and drafting, frontier model reserved for the call that genuinely needs it, a routing layer deciding which is which based on task complexity, cost, and how much you can afford to be wrong.

The frontier labs are saying the quiet part out loud, from their own direction. At a Federal Reserve conference last year, Sam Altman said it looks like the industry is about to deliver on “intelligence too cheap to meter”. He was selling something, as he always is. But strip the salesmanship and he is describing the same abundance I am describing from the operator’s chair, arriving at it from the lab side while I arrive at it from the staffing side. When the person building the models and the person hiring the engineers independently land on “the constraint is no longer the intelligence,” that is worth taking seriously.

The hard part didn't disappear. It moved.

Here is where I part company with the excitement, because cheap intelligence does not make the problem easy. It relocates the problem, and relocation is easy to mistake for solution.

I have spent more than a decade placing engineers, and this pattern is familiar to anyone who has ever scaled a team. The moment labor gets cheaper or more available, the bottleneck does not vanish. It moves up a level, from doing the work to coordinating the people doing the work. A company that suddenly has ten junior engineers instead of two does not have an easier management problem. It has a much harder one, and if it lacks anyone who can actually direct ten people, the extra hands produce more mess, faster.

Cheap agents are exactly this. An organization that can now afford to run fifty agents in parallel has not been handed a solved problem. It has been handed a coordination problem it did not have last quarter. Who verifies the swarm's output. Which agent's conclusion wins when two disagree. How you notice when all fifty are confidently marching off the same cliff because they inherited the same wrong assumption at step one. This is what agent orchestration actually is once you strip the vendor gloss off the term: not a framework you buy, but the judgment about who checks the work, who breaks the ties, and who is accountable when the swarm is confidently wrong. The value does not come from the cheap tokens. It comes from having someone who can architect and supervise what you spend them on.

That is a management problem wearing an infrastructure problem's clothes. Reliability was never a property the model had. It is a property of the arrangement around the model, and abundance makes the arrangement harder, not easier.

What I'd actually do about it

If I were building an AI product right now, I would stop hard-coding my architecture to today's prices and start treating model choice as something my system decides at runtime, not something I decide once.

Concretely. Put a routing layer between your product and any single provider, so when the price-performance frontier moves in three weeks, and it will, you change a config rather than rebuild a system. Make verification a first-class step, not the afterthought you cut when the bill arrives, because the whole point of cheap intelligence is that you can finally afford to check the work. Measure completed work per dollar across the whole workflow, not token price and not benchmark scores, because a weaker model you can run three times with a verifier can beat a stronger one you could only afford once. And hire, early, the person almost nobody is hiring yet: someone who holds software architecture, cloud infrastructure, and agent operations in one head and thinks in blast radius and recovery paths. That person is hard to find right now, which is exactly why finding one before your competitors write the job description is a durable edge rather than a cost.

The honest objection

The sharpest pushback on all this has a name now. It is called tokenmaxxing, and its champions are not fools. Garry Tan of Y Combinator has embraced it, arguing founders should spend aggressively on tokens early because the capability only gets more widespread from here. The bull case is simple: stop being stingy, throw cheap intelligence at everything, and you will be two years ahead of the teams still counting calls.

Half of that is right, and it is the half I have been arguing. Spend the surplus. But the other half is where I get off, and the real world already ran the experiment. Uber did exactly what the bull case recommends. It handed Claude Code and Cursor to roughly five thousand engineers and ranked teams on a leaderboard by how many tokens they burned. Adoption exploded, the budget was gone in four months, and the outside diagnosis was precise: the teams driving the spend were not the teams managing it. That organizational gap, not the price of tokens, was the load-bearing flaw. Spending cheap intelligence is not a virtue in itself. It is a virtue only when each call buys something real: a verification, a retry that catches an error, a task you could not previously automate. 

And I will concede the part I cannot prove. Prices could stop falling. That January doubling could be the shape of things to come rather than a blip. I doubt it, given the trend line. But if I am wrong, it is because intelligence turned out scarcer than it looks, and the teams that built flexible routing layers are still fine while the teams that hard-coded a swarm are not. The architecture I am arguing for survives being wrong in either direction.

Software companies spent the first phase of the AI boom learning to use intelligence efficiently. The next phase belongs to the ones who figure out what to do when they can afford to use far more of it, and who they put in charge of deciding.

Gino Ferrand is the founder and CEO of Tecla, which builds and operates AI systems for U.S. companies and staffs the senior engineering teams behind them, across the U.S. and Latin America. He writes Founder's View, a weekly operator's take on the AI news that actually changes how technology companies build. This piece began as an issue of his newsletter, Redeployed. Talent is everywhere; opportunity is not.