The Cheaper It Gets, The More It Costs

Intro

In 1865, a 29-year-old economist called William Stanley Jevons published a book about coal.

Britain was in the middle of an efficiency boom.

James Watt's steam engine had transformed what a ton of coal could do. His design used a fraction of the fuel the old Newcomen engines burned for the same work. Everyone drew the obvious conclusion. More efficient engines, less coal burned, reserves last longer, crisis averted.

Jevons looked at the numbers and said: no. The opposite is happening.

He was right. As engines got more efficient, Britain burned dramatically more coal, not less.

Not despite the efficiency, because of it.

When a ton of coal did more work, coal became worth using for things nobody would have bothered with before. Pumping water out of deeper mines. Powering mills that had run on rivers. Moving goods that had moved by canal. The efficiency didn't get banked as a saving. It got spent on more.

We named it the Jevons paradox [1]. It has held for a hundred and sixty years across lighting, refrigeration, engines, and data centres.

And right now, every headline you read tells you AI is getting cheaper. That reporting is accurate. It is also exactly why your bill is going up.

The mechanism of the paradox

When the unit cost of something falls, two things happen.

1/ Each existing use gets cheaper.

2/ The set of things worth doing with it expands.

Use cases that were uneconomic at the old price become economic at the new one.

If demand is elastic, and if there are lots of things you'd do with it if only it were cheaper - the second effect swamps the first.

Total consumption rises even as unit cost falls. The saving isn't captured. It's reinvested.

The loop that has held for a hundred and sixty years. The saving is spent, not banked.

Two years ago I argued that better tooling would produce more developers, not fewer. When building software gets cheaper, we don't build the same amount of software with fewer people, we build far more software. That's Jevons paradox - just applied to labour instead of coal.

And it's showing up on invoices now.

The real world

A support chatbot. Nothing exotic. Just a chatbot which could carry out retrieval over a product knowledge base, with a decent system prompt, and had tool access to look up order status.

It passed every eval. The team had run the numbers during the POC and the cost per conversation came out at a fraction of a penny. Rounding error on the P&L. The business case practically wrote itself against the cost of a human handling the same query.

It went to production. The first month's invoice came back at more than twenty times the cost that was modelled.

Nothing had changed. Same model, same pricing, same configuration. No one had misconfigured anything.

The test conversations were just short.

A model has no memory. Every turn in a conversation re-sends the entire conversation so far; the system prompt, the retrieved information, every previous exchange. That's the only way the model knows what you're talking about. This is context.

Turn one is cheap. Turn thirty pays for turns one through twenty-nine all over again.

Run the arithmetic. Say a 2,000-token system prompt and retrieved context, a 100-token user message, a 300-token response. Turn one costs you about 2,100 input tokens. Turn thirty costs about 13,700. Six and a half times more expensive than turn one for an identical question.

Cost doesn't scale with conversation length. It scales with the square of it.

The grey base is what you think you are paying for. The orange band is the conversation history you re-send, and re-pay for, on every single turn.

Then two multipliers land on top. Retrieved chunks pulled mid-conversation stay in the history too, so context grows faster than the message text alone. And POC users ask curated questions, while real users ramble, correct themselves, and change the subject twice.

Why agents are the steam engine

If a chatbot conversation is quietly quadratic, an agent is that same curve with a multiplier bolted on.

A chatbot is one call and one response. An agent is a reasoning loop. It plans, calls a tool, reads the result, reconsiders, calls another tool, and every single one of those iterations carries the accumulated context of everything that came before. One user request becomes eleven model calls. The context window doesn't just grow: it grows on every pass through the loop.

Goldman Sachs models this directly. Their analyst Jim Schneider describes agentic AI as "taking a simple chatbot request and blowing it up tenfold, twentyfold, fiftyfold" [2].

I wrote a while back that generating code was never the revolution. Typing code was always the cheap ten percent, and the ninety percent under the waterline was the expensive part. Both things are true at once, and this is where they meet. Generating tokens is cheap. Which is precisely why Jevons predicts we will generate an absolutely staggering number of them.

Where we are today

AI.cc's 2026 AI API Infrastructure Report, drawn from 2.4 billion API calls across its platform, puts the blended cost of AI down 67% year over year, from $18.40 to $6.07 per million tokens between Q1 2025 and Q1 2026 [3].

Over roughly the same window, the FinOps Foundation's 2026 State of FinOps report found that "73% of enterprises reported their AI costs exceeded original projections" [4]. Uber burned through its entire annual AI budget in four months, with individual engineers running between $500 and $2,000 a month in API costs [5].

Both charts are true at the same time. The price line goes down and the volume line goes up faster.

Every one of those statements is accurate. AI is getting cheaper. Your bill is going up. Jevons would have found none of this surprising.

Where this goes

Goldman Sachs estimates agentic AI could push global token consumption up 24-fold between 2026 and 2030, reaching around 120 quadrillion tokens a month, and by up to 55 times by 2040 if enterprise agents reach full-scale adoption [2].

The IEA reports that electricity demand from AI-focused data centres grew roughly 50% in 2025, against 3% growth in global electricity demand overall, and notes that reasoning and agentic tasks can consume hundreds or thousands of times more energy per query than simple text generation [6]. The coal analogy stops being an analogy somewhere around here.

And it has left the engineering org. Meta named third-party AI token costs as one of the drivers behind a 55% year-over-year rise in quarterly expenses on its Q2 2026 earnings call [7]. This is a board conversation now.

Token economics is an architecture problem wearing a finance costume

The invoice arrives in a format finance recognises. A number, a vendor, a line item, a month-on-month delta. Therefore, it gets handled the way every other line item gets handled: someone is asked to go and negotiate it down.

That instinct is wrong, and expensively so.

In the same AI.cc dataset, organisations running a tiered model architecture, routing each task to the cheapest model that can do it, achieved a median blended cost of $2.31 per million tokens. Organisations routing every workload to frontier models paid $18.40 per million for the same work [3].

That is an 87% gap, and it is the financial consequence of a single architectural decision, usually taken early and rarely revisited.

No procurement negotiation on earth gets you 87% off with the product untouched. An architect does.

That's because the cost was never set at the invoice. It was set at design time. In model selection, in how much context you carry between turns, in how many chunks you retrieve, in whether the agent loop has a hard iteration cap. By the time the number reaches finance, every decision that produced it was made months ago by someone with an IDE open.

Which is also why the metric matters. Cost per token is a number you can always improve by making the system worse; cheaper model, less context, and thinner retrieval.

Value per token is the one that survives contact with the product.

The industry has already half-worked this out, incidentally: "78% of FinOps practices now report into the CTO or CIO organisation, up from 61% in 2023" [4].

What this looks like on AWS

Right-size the model per subtask. The single biggest lever, and the most commonly skipped. Classification, extraction, routing and summarisation do not need your frontier model. Defaulting everything to the most capable model available is the architectural equivalent of heating one room by leaving the oven open.

Prompt caching. Go back to the staircase diagram. That orange band, the conversation history you re-send and re-pay for on every single turn. Caching is the direct structural fix to exactly that. If you take one thing from this post into your architecture review, make it this one.

Batch inference for anything that isn't interactive. Overnight document processing, backfills, bulk classification. You're paying an interactivity premium on workloads where nobody is waiting.

Attribution before optimisation. Application inference profiles let you tag spend to a team, an app, a use case. Without this, Bedrock arrives as one line item and you cannot tell whether you're funding your best product or your most enthusiastic prototype. You can't prioritise what you can't see.

Bound your loops. An agent that can decide to iterate again is an agent that can bill you all weekend. Hard iteration caps, guardrails, timeouts. Treat a runaway loop as a production incident, because it is one.

Tune retrieval. Chunk size is context size is cost. Retrieving twelve chunks where four would do is a decision you make once in config and pay for on every request forever.

Budgets and anomaly detection. Find out in hours, not at month end.

Back to Jevons

Jevons wasn't arguing against efficiency. He wasn't telling Britain to stop improving its engines. And Britain didn't get poorer burning all that coal. It built an industrial economy on it. The consumption wasn't the failure. The consumption was the entire point.

The failure mode was never using more. It was using more without understanding what you were buying.

Cheap tokens are a gift.

Software used to be a fixed cost you paid once and maintained. Agentic software delivery is a variable cost per execution, and almost nobody has rewritten their business case to match.

Do that. Then stop measuring cost per token and start measuring value per token.

Because the price is going to keep falling. And your bill is going to keep rising.

References

[1] Jevons, W. S. The Coal Question: An Inquiry Concerning the Progress of the Nation, and the Probable Exhaustion of Our Coal-Mines. Macmillan, 1865.

[2] Goldman Sachs Research, Decoding the Agentic Economy, May 2026. Analyst commentary from Jim Schneider.

[3] AI.cc, 2026 AI API Infrastructure Report, published 9 May 2026.

[4] FinOps Foundation, State of FinOps 2026, based on 1,192 survey respondents.

[5] Fortune, "Uber burned through its entire 2026 AI budget in four months," 26 May 2026.

[6] International Energy Agency, Key Questions on Energy and AI, April 2026.

[7] Meta Platforms Q2 2026 earnings call, July 2026.