12.10.10 — Agent Economics

When the refund agent was ready to handle all refund requests, Cartwise's finance head asked two questions. What does one resolved refund cost us? And what will the bill be at ten times today's volume? The engineers' first answer, "about five cents a run", turned out to be the wrong number in both directions. It was too high, because they had not turned on caching. And it was far too low as a measure of what a resolved refund cost, because it ignored the runs that failed and went to a person, and the people who approved refunds.

This page works out an agent's economics properly: what you actually pay for, how to calculate it, and the levers that cut it, in the order that gives the most saving for the least risk. Prices quoted are the published Claude prices at the time of writing; they change often, so the habit to learn is the arithmetic, done with current prices and your own measured token counts.

1. What you pay for

A run's bill has more lines than most people expect.

Model tokens, in four kinds, each with its own price:

Model	Input	Cache write	Cache read	Output
Claude Opus 5.5	$4.00	$5.00	$0.20	$20.00
Claude Sonnet 5.5	$2.00	$2.50	$0.20	$10.00
Claude Haiku 5.5	$0.10	$0.125	$0.01	$0.50

All prices are US dollars per million tokens. Input is ordinary input. Cache write and cache read are explained in section 3: writing a prompt's start into the cache costs a quarter more than ordinary input, and reading it back costs a small fraction. Output is everything the model writes, including its thinking, which is easy to forget and often large.

Everything else the run uses: retrieval for search_policies (the vector database and reranker of Chapter 12.9), any paid external APIs, sandbox compute for agents that run code, and the storage for traces.

People. Every run that hands over costs a support person's time, and every approval costs a support lead's. These usually dwarf the model costs, and leaving them out is the most common mistake in agent business cases.

The cost per run, from the API's own counts, is a few lines of code. The cost line in 12.10.2 counted only plain input and output; once caching is on, the cache counts must be priced too:

PRICES = {   # dollars per million tokens: input, output, cache read
    "claude-opus-5-5":   (4.00, 20.00, 0.20),
    "claude-sonnet-5-5": (2.00, 10.00, 0.20),
    "claude-haiku-5-5":  (0.10,  0.50, 0.01),
}

def call_cost_usd(model: str, usage) -> float:
    inp, out, read = PRICES[model]
    return (usage.input_tokens * inp                          # (1) ordinary input
            + usage.cache_creation_input_tokens * inp * 1.25  # (2) written into the cache
            + usage.cache_read_input_tokens * read            # (3) read from the cache
            + usage.output_tokens * out) / 1e6                # (4) output, thinking included

Lines (1) to (4) price the four token counts the API returns in usage. Line (2) uses the 1.25 multiplier for the default five-minute cache. Logging this per call, per run and per agent version is what makes every number on this page measurable rather than estimated.

2. The number that matters: cost per resolved case

A run that fails still costs money, and then a person costs more. So the useful figure is not cost per run but cost per case resolved, counting what happens to the cases the agent does not resolve.

Suppose a run costs $0.04 in tokens, the agent resolves 85% of cases itself, and the 15% it hands over take a support person about ten minutes, which Cartwise costs at $1.50 a case. Then the expected cost of one case is

\$0.04 + 0.15 \times \$1.50 = \$0.04 + \$0.225 = \$0.265

read as "the run's cost, plus the chance of a hand-over times what a hand-over costs". Compare that with $1.50 for every case handled by a person: the agent cuts the cost of a refund case by more than four-fifths. And notice which term dominates. The hand-over rate, not the token bill, decides the economics: improving resolution from 85% to 90% saves $0.075 per case, more than the entire token cost. Any optimisation that saves tokens but lowers the resolution rate is very likely a loss.

Approvals belong in the same sum. If a support lead spends a minute on each approval and 30% of refunds need one, that adds about 0.3 \times \$0.15 = \$0.045 per case, as much as the tokens. That is another reason, beyond approval fatigue, to send only unusual cases for approval (12.10.8).

3. Caching: the biggest free saving

Every call of an agent re-sends the instructions, the tool definitions and the whole history so far (12.10.2). Nearly all of each request is identical to the previous one. Prompt caching lets the provider keep the processed form of a request's beginning, the KV cache of 12.7.6, for a few minutes, and charge a small fraction of the input price when the next request starts with exactly the same tokens.

Turning it on is one argument:

response = client.messages.create(
    model="claude-opus-5-5", max_tokens=4096,
    cache_control={"type": "ephemeral"},        # (1) cache the request up to its latest message
    system=SYSTEM, tools=TOOLS, messages=run.messages)
u = response.usage
print(u.cache_read_input_tokens, u.cache_creation_input_tokens, u.input_tokens)   # (2)

Line (1) asks the API to cache everything up to the end of this request, so the next call, which repeats all of it and adds a little, reads the repeated part from the cache. Line (2) shows whether it worked: on a healthy agent loop, after the first call, cache_read_input_tokens should be most of the input and cache_creation_input_tokens roughly one step's worth of new content.

What it saves

Take the long investigation from 12.10.5: 3,000 tokens of instructions and tools, then 30 steps each adding a 4,000-token result. At the full input price, its 1,830,000 input tokens cost $7.32. With caching, each step reads the previous step's whole input from the cache at $0.20 per million and writes only the new 4,000 tokens at $5.00 per million:

• cache reads: 1,711,000 tokens \times $0.20 per million \approx $0.34
• cache writes: 119,000 tokens \times $5.00 per million \approx $0.60

That makes about $0.94 instead of $7.32, nearly eight times cheaper, for the same agent producing the same answers. On long runs, caching is the largest single saving available, which is why it should be on for every agent from the first day.

What breaks it

The cache matches the exact beginning of the request, token for token. The order is tools, then system prompt, then messages, and a change anywhere invalidates the cache for everything after it. The common ways teams break it without noticing:

• a timestamp or request ID at the top of the system prompt ("Current time: 14:32:07"), which makes every request different from its first line. Put changing values at the end, in the latest message.
• tools in a different order, or added and removed during a run. Keep the tool list fixed and sorted.
• switching models mid-run, since each model has its own cache. Use a different model in a separate sub-call instead (12.10.7).
• editing earlier messages, which is what context clearing does; hence the advice in 12.10.5 to clear rarely and in large batches.
• pauses longer than the cache lasts.

The last one matters for the refund agent. The default cache lasts five minutes, and the refund agent pauses for 40 minutes waiting for a photo and two hours waiting for approval. After each pause, the conversation's cache has expired and its history must be written again. The instructions and tools are different: they are identical for every customer, and with requests arriving constantly they stay in the cache permanently. Pricing customer A's five-step run from 12.10.2 with all of this gives about $0.038 instead of $0.053. That is a real saving, but much smaller than on the long investigation, because short runs with long pauses spend proportionally more on writing.

A one-hour cache is also available, with writes at twice the input price instead of 1.25 times. For the refund agent it does not pay: one pause is shorter than an hour, but the extra cost of every write outweighs what that one saves, and the two-hour pause outlasts it anyway. Whether a longer cache pays depends on how long your agent's pauses actually are, measured, not guessed.

And the bill shifts to output

With caching on, look at what is left of customer A's $0.038: about $0.018, nearly half, is output, the model's thinking and replies. Before caching, output was a third of the bill; after caching, it is the biggest single line. So the next levers are about producing less.

4. Producing less: effort, thinking and replies

Effort. Current Claude models take an effort setting (low, medium, high and above) that controls how thoroughly the model works: at lower effort it thinks less, makes fewer and more consolidated tool calls and writes more briefly. It replaces the older practice of setting a thinking token budget. Set it per task and measure: for a routine refund, medium may resolve as many cases as high with noticeably less output.

response = client.messages.create(
    model="claude-opus-5-5", max_tokens=4096,
    thinking={"type": "adaptive"}, output_config={"effort": "medium"},
    cache_control={"type": "ephemeral"},
    system=SYSTEM, tools=TOOLS, messages=run.messages)

Replies. Ask for concise customer replies in the system prompt, and set max_tokens to a sensible ceiling. Structured outputs from 12.8.3 are naturally short.

The rule from section 2 governs all of this: lower effort is worth it only if the resolution rate holds, which is exactly what the evaluation set of 12.10.9 measures. Run it at each setting and pick the cheapest one that keeps pass^k where it was.

5. Choosing the cheapest model that does each job

The price table spans a factor of 40 between Haiku and Opus. Most of an AI system's work does not need the most capable model, so sending each job to the cheapest model that does it well is the next largest saving, the cascade idea from 12.3.5 applied to models. The router of 12.10.6 is the natural place.

Cartwise's numbers for "Ask Cartwise", at 100,000 conversations a day, show the effect. In the first design, every conversation went through an Opus agent loop, at about $0.06 a conversation uncached: about $6,000 a day. In the routed design:

Path	Share	Model	Cost each	Per day
Router	100%	Haiku	$0.0001	$11
Delivery questions	70%	Haiku workflow	$0.0003	$21
Refund agent	20%	Opus, cached	$0.038	$761
To a person	10%	none	staff time	n/a

The model bill falls from about $6,000 to about $790 a day, and almost all of what remains is the refund agent, the one path that genuinely needs the strongest model. The delivery questions cost almost nothing because they were never an agent's job: a tracking lookup and a short reply.

Other ways to put cheaper models to work: run subagents on a mid-range model under a strong lead (12.10.7); use a small model for the faithfulness check and output reviews of 12.9.4 and 12.10.8; and for a very high-volume narrow task, consider a fine-tuned small model (Chapter 12.11). Each choice is checked the same way: does the cheaper model hold the resolution rate on the evaluation set?

6. Taking fewer steps

Every step re-reads the context and writes output, so fewer steps is cheaper as well as more reliable, the arithmetic of 12.10.1. The levers come from earlier pages:

• Task-shaped tools (12.10.3): send_replacement instead of five endpoint calls.
• Workflows for the predictable parts (12.10.6): the invoice agent's 8 to 15 steps became one model call.
• Clear instructions on what to do first: October's prompt change in 12.10.9 added four wasted steps per run by dropping one sentence.
• Small tool results, which shrink every later step's input.
• Letting the model chain tools in code. When an agent must make many tool calls whose intermediate results it does not need to read, such as checking stock for each of forty items, some APIs let the model write a short script that calls the tools itself and returns only the final answer. Anthropic calls this programmatic tool calling: the script runs in a sandbox, the forty results stay inside it, and only the summary enters the model's context.

Parallel tool calls deserve a note. Running independent calls at once, as in 12.10.7, saves time, not tokens: the same work is done, only sooner.

7. Paying half for work nobody is waiting for

Some agent work has no customer waiting: the nightly evaluation run of 12.10.9 with eight trials per case, re-checking a month of supplier invoices, summarising yesterday's reviews. For these, providers offer batch processing: you submit many requests at once and collect the results within hours, at half the normal price. With Anthropic's Message Batches API, the discount applies to every token, including cache reads and writes, so it stacks with caching.

batch = client.messages.batches.create(requests=[
    {"custom_id": case.name + f"-trial{t}",
     "params": {"model": "claude-opus-5-5", "max_tokens": 4096,
                "system": SYSTEM, "tools": TOOLS, "messages": case.first_messages}}
    for case in EVAL_CASES for t in range(8)])

Each request carries a custom_id so its result can be matched up later, and params is exactly what would have been passed to messages.create. A multi-turn agent run is not a single request, so batching suits the single-call steps of evaluations and offline workflows rather than whole conversations; the harness can submit each round of model calls as a batch when nothing is in a hurry.

8. Budgets, so that a bad day stays a bad day

Optimisation lowers the typical cost; budgets cap the worst one. They work at three levels.

Per run: the step and cost limits of 12.10.2, set from measured runs, so a looping run stops at a known maximum.

Per customer and per day: a cap on runs or spend per customer, which stops one abusive account (or one buggy integration retrying endlessly) from running up the bill.

Per agent: a daily spend alert at, say, 150% of normal, and a hard ceiling that sends new work to people. Together with the circuit breaker of 12.10.8, this is what keeps an unexpected failure from becoming an expensive one.

And watch the right statistics. Average cost per run hides the runaways. Track the p95 and the maximum, per agent version, as 12.10.9 recommended, because a change that leaves the average alone but doubles the p95 is usually a new loop.

9. Cartwise's levers, in order

Lever	Effect on cost	Risk to quality
Prompt caching	up to ~8× on long runs	none
Route easy work to cheap models	$6,000 → $790 a day	measure per path
Workflows instead of agent steps	one call instead of many	lower (checks in code)
Smaller tool results, task-shaped tools	fewer, cheaper steps	lower (fewer failures)
Lower effort where it holds	less output	measure
Batch for offline work	half price	none
Budgets and alerts	caps the worst case	none

The order matters. The levers at the top cost nothing in quality, and several improve it, so they come first and stay on. Levers that trade quality for cost, such as smaller models and lower effort, come after, each one kept only if the evaluation set shows the resolution rate holding. That is the economics of section 2 in practice: a token saving that loses one resolved case in a hundred is a bad trade.

What the interviewer will push on

"How would you estimate the cost of an agent?" Measure tokens per step by kind (input, cache write, cache read, output with thinking), multiply by current prices, sum over the steps of real runs, and report distributions. Then convert to cost per resolved case by adding hand-over and approval costs, which usually dominate. The tell is that last step.

"Why do long agent runs get expensive, and what fixes it?" Each call re-sends the growing history, so total input grows with the square of the run length. Prompt caching makes the repeated part cheap (about eight times cheaper on a 30-step run); then fewer steps, smaller results and clearing in batches.

"What breaks prompt caching?" Anything that changes the start of the request: timestamps at the top, reordered or changing tools, switching models, editing earlier messages, and pauses longer than the cache lasts.

"How do you choose between models in an agent system?" Route each job to the cheapest model that holds quality on the evaluation set; reserve the strongest model for the steps that need judgement. Show the arithmetic, as in the routing table.

"Would you lower the effort setting to save money?" Only if the evaluation set shows resolution holding, because the hand-over cost of one extra failed case outweighs many runs' worth of saved tokens.

Something to volunteer: after caching, output becomes the largest part of the bill, because thinking is billed as output. That is why effort settings and concise replies matter more than people expect, and it is a sign you have looked at a real cost breakdown rather than a price list.

Recall

• An agent's bill: input, cache writes, cache reads, output (thinking included), plus retrieval, tools, sandboxes, traces, and people. Compute it per call from usage, log it per run and per version.
• The number that matters is cost per resolved case: run cost plus hand-over rate times hand-over cost, plus approvals. The hand-over rate usually dominates, so never save tokens at the expense of resolution.
• Prompt caching reads an unchanged request start at a small fraction of the price: a 30-step run drops from about $7.32 to $0.94. It breaks on any change to the start (timestamps, tool order, model switches, edited history) and on pauses longer than the cache lasts.
• After caching, output is the largest line: tune effort, ask for concise replies, measure.
• Route each job to the cheapest capable model; Cartwise's routing cut about $6,000 a day to about $790. Take fewer steps with task-shaped tools, workflows and small results; let the model chain tools in code when intermediate results need not be read.
• Batch offline work at half price; cap runs, customers and agents with budgets; watch p95 and maximum cost per version.

Self-test: With a run cost of $0.04, a 10% hand-over rate and $1.50 per hand-over, what is the cost per resolved case, and what if resolution improves to 95%? Why did caching save so much less on customer A's refund than on the 30-step investigation? Name two changes to a system prompt that would silently break caching.

Next: 12.10.11 How Real Agents Are Built — the chapter closes by looking at how production agents are actually put together, from coding agents to hosted agent platforms, and by tracing the refund agent through every piece of this chapter at once.