Measure context costs before building an agent steward

A component that accounted for 17.5% of injected text accounted for only 0.4% of modeled total Claude usage.

That difference changed my recommendation. I had been looking at a long follow-up block and thinking about removing it to save quota. A later study priced the tokens each call actually carried, including cached reads and writes, then asked what share of usage each proposed change could affect across both hosts. The follow-up block was a small fleet-wide lever. Reducing boot context across many workers was larger.

The question was whether to build a context Steward for long agent sessions: rotate sessions, retain memory, mask older tool output and recover the working set. Before adding another service, we needed to know which roles reached the relevant limit, what the existing harness already handled and which optimization reached the most usage without damaging task quality.

Build the denominator before the design

We scanned successive transcript and runner-log cohorts. The first dev-only study covered 21,962 files. The combined dev and dev2 snapshot covered 28,825. Experiments 32 and 33 used 28,890; experiment 34 used 28,895. These are successive historical corpus versions, not independent replications of a single 24-hour experiment. The file count does not mean every file was a separate session or outcome.

The combined snapshot showed median interactive boot context around 65.3K tokens and headless boot around 53.9K. The estimated excess reacquisition after compaction was 65K at the median and 80K at the 75th percentile. That excess-growth measure is not necessarily a literal count of files reread; Claude Code reattaches file references, and much of the work happens through Bash.

A proposed 15K boot floor did not describe the typical interactive session. The measured boot context had several sources: tool definitions, project instructions, session hooks and the system prompt. In the larger headless and subagent cohorts, project instructions and deferred-tool listings occupied meaningful parts of the starting context. That points to an operating question before a model question: what does each worker role need to know on every task?

Compare each lever against all usage

Experiment 34 estimated the marginal cost of a context token from observed uncached, cache-read, five-minute cache-write and one-hour cache-write proportions. The study used notional units, not invoices. Account identities were absent, so usage is pooled by host rather than correctly allocated to subscription accounts.

The model estimated that rotating interactive coordinator sessions at a 200K threshold with 75th-percentile reacquisition would reduce 4.8% of all Claude usage. Trimming 10K tokens from every session’s boot context modeled 4.5%. A 20K boot reduction modeled roughly 9%. Removing the per-prompt follow-up block modeled 0.4% of total usage, even though that block represented 17.5% of injected-text attribution. It amounted to 2.6% of interactive usage.

Those results are not interchangeable with percentages of text. Cache reads are cheaper than uncached input in the model, and compaction removes old context. A fixed block can look large in a transcript while contributing little to total modeled cost. Conversely, a modest boot reduction may reach almost every worker call.

Here is the shape of the calculation:

marginal = (
    uncached_tokens
    + 0.1 * cache_read_tokens
    + 1.25 * five_minute_write_tokens
    + 2.0 * one_hour_write_tokens
)

saved = removed_tokens_per_call * marginal_cost_per_context_token
fleet_share = sum(saved_for_calls) / sum(modeled_usage_for_all_calls)

This sketch explains the denominator, not a billing API. The actual study computes from recorded request usage and its stated weights. It models usage under assumptions; it does not prove cash savings or quota reduction for a particular account.

Role mix changes the build-versus-buy answer

In experiment 34’s modeled mix, subagents represented 59.6% of usage, headless workers 24.1%, interactive coordinators 16.0%, and runner Claude workers 0.3%. A 10K boot trim reached a much broader base than a coordinator-only rotation. The largest boot opportunities were in subagent and headless contexts, not just the visible long-running coordinator.

Rotation was sensitive to reacquisition cost. In a separate interactive-only simulation, the 200K policy modeled 30.1% lower cost than automatic compaction at the estimated 75th-percentile reacquisition cost, but only 8.7% lower at twice that cost. Those are interactive-cohort comparisons, not fleet-wide usage shares. A separate Codex simulation modeled ordinary sessions 9.4% more expensive under the same 200K policy, even though one long coordinator benefited. The policy did not generalize across session types.

A separate simulation showed ordinary Codex sessions getting 9.4% more expensive under a 200K rotation policy at a 0.1 cached-read weight, while a very long coordinator thread benefited. The policy did not generalize across session types. One global rotation threshold would optimize the wrong sessions and add handoff work where native compaction already cost less.

We also measured a quality proxy. Tool-error rates were lower in higher context bands, but task mix and early exploration can explain part of that curve. After compaction, the observed error rate rose from 2.03% in the 30 calls before to 3.55% in the 30 calls after for interactive Claude; Codex moved from 3.86% to 5.09%. These are observational rates, not a controlled measure of recall or intelligence. At task boundaries, the Claude reset sample had only 14 windows; it cannot settle whether a carefully designed handoff is safe.

Quota events were present: 29 of 158 interactive Claude Code sessions in one recorded cohort hit a usage limit. That establishes that the constraint occurred for those sessions. It does not turn a weighted simulation into a measured invoice or prove that a Steward would prevent the events.

Measure the cheap change first

The recommendation became: do not build the Steward for a generic quality promise. First map boot context by worker role, remove repeated material that the role does not need, preserve hard constraints and remeasure accepted outcomes. Keep the Steward off unless quota still binds after those simpler changes.

If a selective rotation test remains justified, hold task outcomes independently and replay the same acceptance set before and after handoff. Count recovery errors and owner rework as part of the intervention. A context reset is not free just because it uses fewer input tokens.

This is a useful CTO decision pattern beyond agent context. A local optimization percentage needs an exposure denominator. Before funding a new service, compare its fleet-wide reach, maintenance cost, quality risk and the smaller change that might achieve the same business result.

Start with one worker role. Measure its boot sources, cache mix, actual limit events and accepted task outcomes. Then test the smallest change that reaches enough of the workload to matter.

How this was verified

The study records are pinned at gorunner commit ce4719139365e81ee1555c184cfb7a25d1776eb7; the RESULTS-20261021.md and STATUS-20261021.md Git objects matched their dossier SHA-256 values. Figures above are modeled usage estimates, observational log comparisons and historical cohort counts as labeled. This writing pass did not rerun the full-corpus scan, make provider calls, read account-level billing, or run a controlled handoff-quality experiment. The 28,895-file corpus is historical, not a claim that all files were generated in the discovery window.