The math behind agent costs
An agent loop repeats: the model reads the context, emits a tool call, your code runs the tool, and the result is appended. At step i the model reads:
context(i) = system_prompt + tools + task + (i − 1) × (output_per_step + tool_result_per_step)Summed over N steps, input tokens grow with N². Doubling the number of steps roughly quadruples input cost, which is why step count and tool-result size matter more than almost any other design choice.
How caching is modelled
- Step 1: the system prompt is read from cache if it's already warm from earlier runs. Otherwise the whole prompt is written to the cache.
- Every later step: everything up to the previous step is a cache hit, and only the newly appended output and tool result are written.
- Anthropic charges 1.25× input for 5-minute cache writes and 0.025–0.1× for reads, depending on the model. OpenAI and Gemini cache automatically, with no write premium.
Design choices that change the bill
- Keep tool results small. Return IDs, snippets, and summaries. Let the agent ask for more.
- Batch tool calls. Parallel tool calls in one step avoid an extra round trip that re-reads the whole context.
- Clear or compact old context. Dropping stale tool results or summarizing history stops the quadratic growth.
- Route by difficulty. A frontier model for planning with cheaper sub-agents for reading and extraction is often the best cost-quality mix.
- Measure cost per completed task. A cheaper model that needs twice the steps or fails more often is not cheaper.