Your Agent's Real Unit Cost Is Per Completed Task, and You Are Probably Not Measuring It

There is a spreadsheet in most companies running agents in production that multiplies expected requests by a per-million-token price and produces a monthly figure. That spreadsheet is almost always wrong, usually by a multiple rather than a percentage, and it is wrong in a direction that makes the cheapest-looking model the one you regret.

Per-token price is an input to your cost, not a measure of it. The number that decides your invoice is tokens per completed task, and there are four terms in that calculation that the spreadsheet usually omits.

Start with the shape of the workload

An agent is not a chatbot with more steps. It has a structurally different cost profile, because it re-sends its accumulated context on every tool call. The conversation grows as the loop runs, and you pay for the whole accumulated thing again on each turn.

The consequence is an input-to-output ratio that surprises people the first time they measure it. A representative fifty-turn coding session runs on the order of a million input tokens against forty thousand output tokens. That is roughly twenty-five to one, with input accounting for something like 85% of the total cost.

Gartner's March 2026 analysis put agentic workflows at five to thirty times the tokens per task of a simple chatbot query. That range is wide because it depends entirely on how many turns your loop runs and how fast your context grows, but the direction is consistent, and it is why cost estimates built from chat-shaped assumptions come in low by so much.

Two things follow immediately. First, output pricing matters less than you think for agent workloads, even though it is the higher headline number. Second, anything that affects how much input you re-send, cache hit rate above all, is a first-order cost lever rather than a tuning detail.

Term one: the success rate divisor

This is the term I see left out most often, and it is the one that most often reverses a decision.

A failed task bills at full price. The model ran, the tokens flowed, the tool calls executed, and at the end the output was wrong or the loop gave up. You pay for that exactly as you pay for a success.

So the unit you care about is not cost per attempt, it is:

cost per completed task = cost per attempt / success rate

Run that with numbers. Suppose a cheaper model costs $0.40 per attempt and completes 70% of tasks. Effective cost is $0.40 / 0.70, about $0.57 per completed task. A more capable model costs $0.60 per attempt, 50% more, but completes 85%. Effective cost is $0.60 / 0.85, about $0.71.

In that example the cheap model still wins, and I am showing it that way deliberately, because the point is not that expensive models always win. The point is how quickly it flips. Hold the prices and move the cheap model's success rate to 55%: now it is $0.73 against $0.71 and the ordering has reversed. A fifteen point swing in reliability, entirely ordinary between model tiers on a hard workload, is enough to invert a 50% price difference.

These are illustrative figures, not measurements of any specific model. The method is the transferable part: you cannot compare two models on price without their success rates on your workload, and that means you need a definition of success you can compute automatically over a sample of real tasks.

Term two: cache hit rate

Given that input is around 85% of an agent's bill, the fraction of that input served from cache is close to being the whole game.

Prompt caching prices repeated reads of a stable prefix at a fraction of the uncached rate. In an agent loop the stable prefix is large: the system prompt, the tool definitions, and the accumulated earlier turns. Keeping that cached across a long session is the difference between paying full price for a million input tokens and paying full price for the marginal tokens only.

The things that silently destroy your hit rate deserve to be known by name. Switching models mid-conversation starts a cold cache, because caches are per-model. Injecting a timestamp or a request ID near the top of the system prompt invalidates everything after it on every call. Reordering tool definitions between requests does the same. Rewriting the system prompt per user, rather than keeping a shared stable prefix and putting the user-specific part later, prevents the prefix from ever being shared.

None of those show up as errors. They show up as a bill that is two or three times your model, with a per-token price that matches the pricing page exactly. If you do not have cache hit rate on a dashboard next to spend, you cannot see the largest lever you have.

Term three: step count, which is what token efficiency actually means

The headline claim on the 2026 launches was not a benchmark score. When OpenAI shipped the GPT-5.6 family on 9 July, the number Sam Altman led with was that Sol is 54% more token efficient on agentic coding tasks. That framing has become standard because it maps to the bill more directly than a benchmark does.

Here is why efficiency compounds rather than adding. A model that takes fewer steps to reach the same outcome does not just save the tokens of the steps it skipped. Every step it skips is also a step whose output never enters the context that all subsequent steps re-send. Fewer steps means a smaller context at every later turn, and less input on each of them.

The effect compounds on a growing context rather than adding to it, and it is why a more expensive model that solves a task in twelve steps can cost less per completed task than a cheaper one that takes twenty. It is also why per-token price is such a poor proxy for agent cost: it prices the atom while step count prices the molecule.

Anthropic's Opus 5, released 24 July 2026 at $5 per million input and $25 per million output, held the same pricing as Opus 4.8 before it while improving capability. Read through the cost model above, a capability improvement at flat pricing is a price cut, because it moves both the success rate divisor and the step count.

Term four: the costs that are not tokens

The three terms above are all denominated in model spend, easy to put in a spreadsheet and easy to over-focus on. The remaining costs are larger and harder to see.

Retries and reruns. Not the model's internal retries, the human ones. Someone looks at the output, decides it is wrong, and runs it again with a better prompt. That second run is a full-price attempt, and if your success metric is computed after the human gave up rather than after the first attempt, your measured success rate is flattered.

Review time. If a human checks every output before it is used, their time is part of the unit cost, and it is usually the dominant part. Published figures for AI coding assistance land around $13 per developer per active day and $150 to $250 per developer per month, with heavy automation reaching $500 to $2,000 per engineer per month. Compare those to a loaded engineering hourly rate and it is clear which side of the ledger to optimise. A model that costs 30% more and cuts review time in half is not a close call.

The cost of a wrong action. For read-only agents this is zero and you can ignore it. For an agent that writes, sends, or spends, the expected cost of a wrong action belongs in the model, and it does not scale with tokens at all. This term is why per-token thinking breaks down completely for action-taking agents: the entire risk profile is orthogonal to the pricing page.

What to actually instrument

The model above is only useful if you can compute it, and computing it needs three things your traces probably do not have yet.

A task boundary. Not a request, a task. Every model call needs a task ID so you can sum cost across an entire loop and attribute it to one unit of work. Without this you have a per-call cost distribution and no way to roll it up.

An outcome per task, computed automatically. Some tasks have a checkable definition of success: tests pass, schema validates, the extracted value matches a known record. Where you have that, record it. Where you do not, sample and label by hand periodically, because an approximate success rate measured on a hundred tasks beats no success rate measured on all of them.

Cache hit rate per call, alongside token counts. Most providers report this. Get it into the same store as everything else so you can see spend and hit rate on one chart.

With those three, the cost model computes itself, and you can answer the questions that actually come up. Which step class is consuming the most tokens. Whether last week's prompt change moved cost per completed task or just moved cost. Whether the cheaper tier is cheaper once you divide by success rate.

The uncomfortable conclusion

If you run this analysis honestly, a common result is that the model is not where your money is going. The bill is dominated by context you are re-sending because your cache is cold, steps you are running because the loop does not know when to stop, and human review time you are paying because nobody trusts the output enough to skip it.

All three of those are engineering problems with engineering fixes, and none of them get better by switching to a cheaper model. Switching to a cheaper model usually makes the third one worse.

The real argument for measuring cost per completed task rather than cost per token is not that it produces a more accurate number, though it does. It is that the number points at the thing you should fix.