AI operating cost and harness design / Field essay 43

What the subscription meter actually charged for tokens.

API list prices were a poor proxy for subscription quota. Isolated token-class experiments measured reads, writes and output separately and changed the optimization ranking.

Published

AI OPERATING COST MEASUREMENT AND QUOTA ECONOMICS
FIELD ESSAY 43 / AI OPERATING COST AND HARNESS DESIGN

What the subscription meter actually charged for tokens

I stopped arguing about whether a coding subscription quota behaved like API pricing and measured it. The result changed how I ranked cache reads, writes, and generated output.

The question came up after small optimization experiments seemed to produce almost no extra capacity. A gateway vendor claimed a two-times multiplier, but our tests had not measured the mechanism behind that claim. Before deciding the idea was wrong, I wanted to know what our own subscription meter charged for each token class.

I isolated an idle account so other work could not move its meter. I read the five-hour utilization header from a one-token call, then ran separate phases: fresh prompts to create cache writes, forks of a fixed synthetic prompt to create cache reads, and repeated answer generation to create output tokens. The meter moved in whole percentage points, so every rate had uncertainty of roughly one point. That resolution is coarse, but the token volumes were large enough to distinguish the phases.

The measured rates changed the ranking

One point of this account’s five-hour window corresponded to roughly 19.8 million cache-read tokens, with a range of about 17 to 24 million. A one-hour cache write was about 360,000 tokens per point, with a 310,000–430,000 range. A five-minute write was about 430,000 tokens per point, with a 310,000–540,000 range. Generated output was about 91,000 tokens per point, or 83,000–101,000.

Reads counted, but much less heavily than API list prices would suggest: about 0.36 times the API-weighted rate in this test. Output counted more heavily, about 1.6 times. The five-minute write band overlapped the one-hour band because the meter’s resolution was coarse; the sample did not justify a crisp claim that one write class always costs a fixed fraction of the other.

The phase totals cross-checked the fitted rates. One fresh one-hour write of 359,500 tokens used one point. One hundred reads totaled 36 million read tokens plus 66,000 write tokens and used two points. Three hundred reads totaled 107.9 million plus 197,000 writes and used six points. Six fresh one-hour writes totaling 2.157 million tokens used six points. One hundred long answers produced 952,000 output tokens and used eleven points. Six five-minute writes totaling 2.160 million tokens used five points. The predicted and observed phase deltas agreed within the meter’s whole-point steps.

Why this was a better test than another gateway trial

The key was to isolate token classes before comparing them. A mixed workload can make an optimization look ineffective because reads, writes, and output are charged differently. I held the account out of normal work, used a stable prompt for read bursts, fresh sessions for writes, and a small answer task for output. Each phase had a direct meter delta and token total.

I also kept the experiment small enough to fit in one five-hour window. The account started mostly idle; the test itself used 32 percentage points of that window. I did not infer a universal rate from the provider’s pricing page. These are observed rates for one subscription account and model configuration, with one-point measurement granularity.

That distinction mattered for the gateway claim. We had tested prompt trimming and compression, but those did not answer whether cache reads moved the subscription meter cheaply. The direct experiment did. It showed reads were not free, but they were far cheaper than API-weighted estimates. It also showed output consumed quota much faster per token than cache reads. The result did not support a two-times gain for this fleet under its measured mix.

A useful meter is a model of the workload

I now treat quota optimization as a workload-accounting problem. First separate the meter into token types. Then measure the actual mix by role: interactive sessions, headless runs, and subagents did not have the same distribution. Only after that should I choose between shortening prompts, extending cache lifetimes, reducing output, or changing the routing layer.

The companion analysis of our worker traffic found a second result: subagents were rewriting large prefixes after cache expiry. That mechanism is related, but the two questions are different. This article measures what the subscription meter charged per token class; the worker analysis measures why cache writes repeated after long waits. Keeping those measurements separate prevents one result from standing in for the other.

If a vendor promises a multiplier, ask which meter, model, account, token class, and workload mix produced it. Then design a test that isolates those variables and includes the meter resolution in the uncertainty. A precise-looking ratio from a coarse meter is still an estimate.

Calibrate a meter before changing routing

A subscription utilization meter is not a bill with perfect precision. It rounds, covers a rolling window and may combine token classes with different weights. I isolated an idle account so unrelated sessions could not move the reading, then ran separate phases: create cache writes, fork a fixed synthetic prompt to create reads, and generate repeated outputs. The changing meter values were coarse, so each inferred rate needed a range rather than a false exact price.

The discipline matters more than any one rate. If cache reads dominate, shortening the prompt or changing models may not be the first lever. If cache writes dominate, the location of per-session data and long waits may matter more. Output cost may move differently again. These are separate axes; collapsing them into one “tokens used” number hides the mechanism and encourages the wrong optimization.

Separate the meter from the workload

The right calibration experiment changes one thing at a time. I use the same bounded task and prompt, then create read-heavy, write-heavy and output-heavy phases. I record the account meter before and after each phase, repeat the run, and keep the session type visible. If the provider rounds usage or batches updates, a single short sample can produce a misleading ratio. The goal is a stable comparison, not an exact universal exchange rate.

This is also why I do not reuse API list prices to explain a subscription quota. An API invoice and a subscription usage meter can account for cached reads, writes and output differently. The meter is the operational constraint the team experiences; API prices are a different purchasing model. I label which one a measurement describes and avoid converting one into the other without an explicit contract.

After calibration, routing can use the measured category that dominates a particular workload. A change that reduces output but causes repeated full-prefix writes may lose overall. A change that saves reads but creates more retries may also lose. So the dashboard needs both the meter delta and the quality result: accepted task, review outcome, retries and handoff work. The optimization target is the cost of an accepted result under the actual quota system, not the prettiest per-token number.

Make the measurement repeatable

I record the account meter at a consistent point before and after each workload, note any reset window, and keep background sessions out of the calibration run. Otherwise a quota change from another worker can be mistaken for the cost of the experiment. When the meter is coarse, I increase the workload until the delta is visible and repeat it rather than reporting false precision from a tiny sample. The resulting ratio is useful for ranking choices under that account’s terms; it is not a conversion factor that should be copied into another organization’s budget spreadsheet.

I keep each measurement tied to its account, time window, model and experimental phase. I rerun the phases independently, state the meter resolution, and avoid comparing measurements from different plan windows as if they were invoices. Then I route based on cost per accepted outcome, with quality checked separately. If a change saves usage but increases review failures, it may be a more expensive workflow. What token class does your current model-routing dashboard fail to distinguish?

How this was verified

I checked the experiment protocol and phase totals in the pinned report. Rates are approximate, account-specific measurements from a meter with one-percentage-point resolution. No third-party gateway result is presented as independently verified.

CONNECTED READINGForward and back
Keep reading

The answer deserves
another experiment.

Back to the journal