AI API Pricing Is Broken: Per-Token Billing Charges You for the Wrong Thing

Per-token pricing is the wrong unit for AI APIs. It charges you for effort, not for results — for every token a model burns on the way to an answer, including the tokens you never see, the retries that failed, and the words a rambling answer didn't need. The fix is not a cheaper token. It is a different unit: price per finished task, not per token spent getting there.

That sounds like a small accounting complaint. It isn't. The unit you pay by decides what you optimise for, what you fear, and what your provider is rewarded for. Right now, all three point the wrong way.

You pay for the journey, not the arrival

Think of a taxi ride. You pay for arriving, with the meter as a rough proxy for distance. Now imagine a taxi that charged per turn of the wheels — extra when the driver took a wrong turn, circled the block, or sat thinking at a junction with the engine running. That is how an AI API bill works today.

Every call is metered in tokens, and a token is not a unit of value. It is a chunk of text, roughly part of a word. You are billed for the prompt you send, for the answer you receive, and for everything the model generates in between — including work that produces nothing you can use. A call that fails halfway still costs you the tokens it consumed. An agent loop that retries a step five times bills you five times for one job. A model that answers in 400 words costs roughly twice as much as one that answers the same question correctly in 200. The meter runs on effort, and effort is the one thing you cannot see or control in advance.

This was tolerable when an API call was one prompt and one answer. It is not tolerable now that a single user request can set off a chain of dozens of calls, each feeding the next, with the total unknown until the chain stops. Our bill-diagnosis checklist walks through the usual ways a bill runs away, and nearly all of them are the same story: a loop nobody capped, a prompt nobody trimmed, a retry nobody counted. Per-token pricing turns every one of those slips into money, silently, at machine speed.

Exhibit one: the tokens you never see

The clearest proof that the token is the wrong unit is the reasoning token. When a reasoning model works through a hard problem, it generates an internal draft — its working-out — before producing the answer you actually read. OpenAI's own documentation says those reasoning tokens are not visible through the API, but they still occupy context space and are billed as output tokens, at the output price.

Read that again: the most expensive tokens on the bill are the ones you are not allowed to read. The same documentation notes that a model may generate anywhere from a few hundred to tens of thousands of reasoning tokens for one problem. You asked for a short answer and paid for a hidden essay you will never see.

Reasoning effort settings make the absurdity explicit. Turn the effort dial up and the same question, with the same correct answer, costs more — because the model was allowed to think longer. The answer didn't change. The value didn't change. Only the amount of invisible work changed, and the meter charges for invisible work at the highest rate on the price sheet. A unit that bills you more for the same answer, depending on a dial you set blind, is not measuring value. It is measuring its own process.

Exhibit two: the same answer at five different prices

If the token were a real unit of value, the same answer would cost the same. It doesn't — not even close. The price of an identical response swings wildly depending on plumbing choices that have nothing to do with what you received.

Send a prompt whose beginning matches a cached prefix and, on Anthropic's published terms, those tokens are read back at a tenth of the standard input price. Write a new cache entry and the same tokens cost a quarter more than standard on a five-minute lifetime, or double on a one-hour lifetime. Send the work through a batch queue instead of in real time and every token is discounted by half, because you waited. Same model, same prompt, same answer — and the bill can differ several-fold based on timing, cache state, and queue choice.

None of these discounts is a bad thing. Caching and batching are genuinely useful, and providers deserve credit for pricing them. But notice what they reveal: the price already depends on everything except the value of the answer — on whether you waited, whether your prefix matched, whether your cache was warm. The token is not a unit of output. It is a unit of compute with a loyalty scheme bolted on, and every discount is an admission that the headline meter doesn't reflect what the work is worth.

Exhibit three: verbosity pays, and that should worry you

The deepest problem is the incentive the meter creates. Under per-token pricing, a provider earns more when its model says more. A model that answers in a tight paragraph bills less than a model that pads the same answer into five paragraphs. A model that reasons at length bills more than one that reasons efficiently. The meter rewards wordiness and wandering, and punishes exactly the qualities you want: brevity, precision, and getting it right the first time.

You would never accept this deal anywhere else. You don't pay a plumber per turn of the wrench, or a lawyer per word of the contract, precisely because you know what happens next: more turns, more words. Yet AI — the tool where output length is most directly under the supplier's control — is priced by the word, and we act surprised when bills grow with verbosity.

Defenders of the token will say it is the only fair meter, because it tracks real compute. That is true and beside the point. Compute is the provider's cost, not the customer's value. A restaurant's costs are ingredients and labour, but it charges per meal, not per gram of flour and minute of stove time — because the customer buys a meal. The token meter confuses the provider's cost ledger with the customer's purchase, and then makes the customer audit the ledger line by line to understand a bill.

What the right unit looks like

The honest replacement is price per finished task. A summarised document. A resolved support ticket. A passing test suite. A completed research brief, with the model allowed to choose how much thinking, retrying, and tool use that takes — because the thinking is the provider's problem, not yours.

Providers are already inching toward it. A batch discount is, in effect, a price for patience rather than a price for tokens. A cached-read discount is a price for repetition. And Anthropic's own cost guidance now frames optimisation in terms of cost per completed task rather than cost per token, because the token stops being informative once work becomes multi-step.

Task pricing would also fix the incentives overnight. A provider selling finished tasks would want its models concise, efficient, and right the first time — because every wasted token would eat its own margin instead of inflating your bill. Verbosity would become a cost to the seller, not a revenue stream. That single flip is worth more than any discount a token price sheet will ever offer.

What to do until the unit changes

The token meter is not going away this quarter, so the practical move is to stop budgeting in tokens and start budgeting in tasks. Measure what a completed job costs you, per feature, and let that number — not the token count — drive your decisions.

That starts with logging the usage object on every call, including the reasoning and cache fields providers already report, so you can see where the meter actually runs. Our cost-tracking middleware tutorial shows how to build that logging in an afternoon. Then put a guard in front of spending so one user or one loop cannot turn a meter into a flood: our per-user spending guard refuses a call before it is made when a budget is already spent. And when you compare options, run your own numbers through our AI API cost calculator — but read the result as cost per task you actually ship, not as a token total to admire.

The token had a good run

The token was a fine unit for the era it was built in, when an API call was a single prompt and a single answer and the meter roughly tracked the work. That era is over. Agents reason invisibly, retry silently, and chain calls you never see — and the meter charges you for a process you cannot inspect, at a rate that changes with plumbing choices you did not make.

Some provider will eventually sell you the finished task instead of the tokens, and take the efficiency risk onto its own books. When that happens, every developer who has ever stared at a surprise invoice will know exactly why they switched. The only question is which provider moves first — and how many more meter-shock bills the rest of us pay while we wait.