Design agent supervision around delivery, intervention and bounded learning

Four timer prompts were waiting for my orchestrator. None of those four had been processed.

At 22:06:50 on October 19, the agent corrected its earlier account of the checks. The distinction was small in the interface and large in the architecture: a timer had put messages in the native queue. It had not independently supervised the turn that was already running.

That is the problem I want another GPT or Claude instance to help solve. A useful supervisor notices that a plan no longer matches reality, brings the right evidence into the active decision, and knows when to leave a long operation alone. It also needs a delivery contract. Otherwise I have built another system that can successfully ask an agent to pay attention without establishing that the agent did.

The experience led me toward three separate mechanisms: a cheap event monitor, a model reviewer invoked for bounded judgments, and an intervention controller with durable identities. Learning improves the decisions inside those mechanisms. It should not let a model invent new powers for itself.

The amount of activity was not the missing signal

I extracted a fixed eight-hour window from the development session named Review orchestrator handoff: 14:15:50 through 22:15:50 UTC. The root transcript recorded 1,312 unique usage responses and 202,066,859 input tokens. Of that input, 199,593,728 tokens were cached: 98.776%. Output was 637,749 tokens. Uncached input was 2,473,131 tokens.

Those totals agree with a second calculation using the cumulative thread counters. They establish recorded usage, not a cash invoice. Cached tokens remain part of input. Reasoning output remains part of output. Adding either again would manufacture a larger number.

The root also made 283 top-level send_message calls and 59 followup_task calls. There were 872 exec calls, whose nested tool invocations are outside that top-level count. All twenty-six root model-context records named gpt-6.1-sol with medium effort. That is not an inventory of the child models or Claude sessions.

There was plenty of communication and computation. The missing guarantee was more specific: did this particular intervention reach the decision that needed to change? A counter that says the agent is active cannot answer that. Neither can eighteen completed conversation turns; they are not eighteen accepted engineering outcomes.

I want supervision measured by recovered useful work and fewer owner interruptions, with its own cost included. Token volume is an input to that account, not its conclusion.

Give observation, judgment and intervention different jobs

The event monitor should answer mechanical questions. Which turn is active? Is a tool running? Is the agent waiting for approval? Which owned intent is overdue? Has the recipient acknowledged that exact intent? Has an externally accepted operation returned a terminal result?

Those questions should not require a model to reread a large transcript every minute. They need typed events, durable timestamps and a small amount of state. Turn and item lifecycle events, tool progress, approval state and queue changes provide different pieces of that state. A metadata-only read is not equivalent to a working event subscription.

The model reviewer has a different job. It can judge whether a plan remains sensible, whether repeated repair attempts are learning anything, whether an integration contract is missing, or whether an apparent stall is legitimate work. I would give GPT or Claude a bounded packet: the intended outcome, relevant source versions, the last verified result, contradictory evidence, current ownership and allowed interventions.

The intervention controller then enforces the result. A review can recommend a steer; the controller still checks the exact turn, authority, outstanding delivery and current critical operation. This separation keeps a persuasive paragraph from becoming unrestricted operational authority.

For difficult reviews, different model families can help expose different assumptions. They do not automatically create independent evidence. If both reviewers receive the same optimistic summary, both can repeat it. An independent reviewer needs the source packet, permission to challenge its framing, and its own controls.

Where the extra model earned its time in our experiment

I also recomputed the earlier planning-versus-repair experiment from its 116 unique raw records across five proxy tasks. The author route label was muse-spark-1.3-contributor. The planning stage used the historical claude-sonnet-5-5 label at low effort; the review stage used glm-5.3 at maximum effort. These are recorded route labels, not current product recommendations.

For low-effort authoring, baseline accepted 9/14 runs at a 73.3-second median. A checklist accepted 9/14 at 87.35 seconds. Plan-plus-tests accepted 12/14 at 100.5 seconds. Review-guided repair accepted 10/14 at 453.05 seconds. At high effort, the accepted counts were 9/15,9/15,12/15 and 10/15; median elapsed seconds were 140.9,142.5,159.8 and 528.3 respectively. Acceptance meant a hidden score of at least 0.9.

That gives me a useful supervision hypothesis: improve the task contract before spending another long repair cycle. It does not prove that a planner always beats review, or isolate planning from its bundled tests. Review can still find a consequential failure a mean score obscures. All raw cost_usd fields were null, so these elapsed-time and acceptance comparisons cannot establish a dollar advantage. The five tasks and correlated repetitions also limit generalization.

A queued message is one state in a longer protocol

Our existing check-in producer already has useful foundations. It takes a per-kind lock, records an intent before dispatch, keeps one outstanding intent and requires an exact acknowledgement ID. Before invoking delivery, it writes dispatch_unknown. If the call times out, the intent stays unresolved instead of being blindly resent.

That ordering matters. If the process crashes after the receiver accepts a message but before the sender records success, a retry can duplicate the intervention. A timeout cannot distinguish that case from a message that never arrived.

I would make the rest of the protocol equally explicit. Proposed means a supervisor wants an intervention. Queued means the transport accepted a queued input. Acknowledged means the recipient recognized the exact intent. Action_started means it began the requested change. Effect_verified means an independent readback supports the outcome. Expired means the intervention is no longer appropriate.

These states must retain their meanings across restarts. The ledger should bind the intent to a thread, an expected turn where applicable, the complete input digest, the producer/configuration version, creation time and deadline. A prefix match on message text is not enough to identify the right queued item. Neither is the latest-looking queue entry.

Routine telemetry can retain these identifiers, event types and hashes without retaining every prompt or tool payload. The evidence packet can point to private source material when a reviewer needs it. Supervision should not create an unnecessary second archive of sensitive development content.

Make the proposed policy executable before making it live

A small offline model helps expose the decisions without touching the running orchestrator:

if (intent.status === 'dispatch_unknown') return 'reconcile';
if (intent.status === 'queued') {
  return now >= intent.ackDeadline ? 'report_overdue' : 'observe';
}
if (snapshot.approvalPending || snapshot.criticalTool ||
    snapshot.externalEffectUnknown) return 'observe_critical';
if (intent.expectedTurnId !== snapshot.activeTurnId ||
    !snapshot.activeTurnId) return 'reconcile_turn';

This is illustrative policy code, not an extract from the installed watchdog. Its twenty-one fixture cases include an allowed steer decision and an allowed interruption decision, alongside stale turns, expiry, unknown delivery, critical tools and withheld authority. It makes zero network calls and deletes zero queue entries. It produces recommendations to prepare an action; it implements neither durable dispatch nor runtime delivery.

That limitation defines the next test boundary. A pure decision function can reject the wrong turn in a fixture while a real adapter accidentally sends an unguarded RPC. Controller tests need the actual transport and restart behavior before activation.

Steering needs a precondition

Codex App Server documents turn/steer with an expectedTurnId. The active turn has to match that precondition. Steering appends input to an in-flight turn; it is not a generic way to start whichever work happens to be available. Interruption is a separate operation against a particular turn. OpenAI App Server documentation.

The practical race is straightforward. A monitor observes turnA, prepares a correction, and sends it after turnA has completed. TurnB may now be doing a different task. An unguarded redirect can land on the wrong decision. A stale-turn rejection should trigger reconciliation, not an automatic retry against the newest turn ID.

A steer should also state the requested change precisely. Stop the broad recursive search and inspect these two directories is more useful than be efficient. Reconcile this deployment receipt before trying another release is more useful than check progress. The model can explain its reasoning, but the controller needs a finite action and an acceptance condition.

At the observed 22:21 status update, a worker was building overdue-inbox escalation into the existing check-in tooling. The reported sequence was steering first, interruption after ignored escalation, with intent identity preserved. Automatic escalation remained disabled pending delivery and duplication tests. That is the state captured by this research; it is not a claim that the full intervention loop was already activated.

Queue promotion is a distributed transaction problem

The installed capability investigation examined Codex 0.163.0 experimental schemas. Queue entries have IDs. There are operations to list, delete and start queued submissions. Steering accepts the expected active turn and input. The investigation found no exposed atomic operation that consumes a particular queued submission and steers its content into the current turn.

That absence creates two tempting but unsafe sequences.

Steer first, then delete the queued copy. If normal queue consumption happens between those operations, the same intent can be delivered twice. Delete first, then steer. If steering fails, the original message is gone; if steering times out, delivery is unknown.

The optional clientUserMessageId is useful identity material, but its existence does not prove deduplication across queue consumption and steering. Queue deletion also lacks an expected-input-hash precondition in the inspected contract. I cannot get exactly-once promotion merely by arranging two separate calls carefully.

True promotion needs an upstream atomic operation or a demonstrated server-side deduplication guarantee spanning both delivery paths. Until then, an existing queued intent should remain intact while its overdue state is reported. A future intent delivered directly by guarded steering can be a distinct mode. It does not establish safe promotion of the old queued message.

There is another evidence boundary here: the read-only capability investigation's initialize-only runtime connection timed out after twelve seconds. It sent no subsequent RPC. Schema availability was established; successful runtime steering, queue-start behavior and promotion semantics were not established by that probe.

Interruption does not undo an external effect

An interrupt can end model work while an external operation has already crossed its acceptance boundary. A deployment service may have accepted a release. A migration may be running. A supplier may have accepted an order. Cancelling the agent turn cannot make those effects disappear.

The supervisor therefore needs the current critical-tool and approval state, not only a no-output timer. Quiet work may be healthy. A blocked approval may require a person. An ambiguous remote call may require receipt reconciliation before anything else. Interrupting and starting a fresh attempt can multiply the very uncertainty the supervisor was supposed to resolve.

I would use interruption when the expected benefit exceeds the cost of cancelling useful work and the external-effect state is understood. The policy needs an owner, a deadline and a recovery path. Repeatedly interrupting a legitimate long build because it has not printed a line is an outage policy disguised as responsiveness.

For each intervention, retain why it was selected and what happened afterward. Did it redirect the intended turn? Did the agent acknowledge it? Did the requested change occur? Did an independent readback confirm it? Did it cancel work that later proved useful? Those outcomes let the rule improve without rewarding the mere act of intervening.

Cancellation has another local boundary

At 22:46, my orchestrator reported that a native grader had run fifteen command pairs despite the same migration-bootstrap error recurring on the candidate side. At 22:48 it corrected the cancellation claim: the installed pause/cancel controls prevented later publication but did not stop an active grading process. Task-bound process cancellation and STOP fail-fast were still being fixed. Those are source-reported observations, separate from the fixed eight-hour usage census and from an independent subprocess-stop receipt.

I need cancellation requested, publication fenced, process signaled, process exited and external effect reconciled to remain separate facts. A stopped publication path cannot establish a stopped process. Infrastructure failure also needs a typed STOP path so the grader does not spend the rest of a known-impossible attempt proving the same bootstrap failure repeatedly. Healthy-work controls still matter: a fail-fast rule must distinguish infrastructure invalidity from an ordinary candidate test failure.

Let models propose rules inside fixed capabilities

We already have a rule implementation that makes this distinction concrete. At the research snapshot, repository configuration contained twenty-seven active and five proposed branch-reconciliation rules, eight active failure-routing rules and three active runner-health rules. The installed runner-health and last-good files matched the repository's version-two configuration.

Those are different evidence claims. Repository rules show configured policy. Matching installed files show that configuration reached disk. A live decision trace would be needed to establish which version a particular running consumer used. A quality evaluation would be needed to show that the decision improved the work.

The action vocabulary is registered in code. A JSON proposal chooses from existing actions; it cannot grant itself arbitrary shell execution by inventing an action name. The transition code also routes owner-origin changes, destructive or fleet-halting actions and new provenance-site grants out of the ordinary approval path.

That gives model learning a bounded surface. A proposal can say: when this observable failure pattern appears, select this already-permitted action, with this recovery and escalation path, supported by these receipts. It cannot redefine the controller's authority simply by writing a more confident rule.

There is a subtle limit worth making explicit. The ordinary promotion code requires a nonempty ticket string. That requirement does not by itself prove that an external authority system validated the ticket, its scope or its issuer. Before calling the full path authorization enforcement, I want an integration test that tries an unauthorized but well-formed ticket and verifies refusal at the actual boundary.

Learn from disagreement before learning from outcomes

A practical rule lifecycle starts with a proposal tied to observed evidence and a predicted failure signature. Replay it against known failures and known successful work. Then run it in shadow mode and compare its recommendations with the real controller decisions.

The disagreement set is valuable. A rule that would have interrupted every quiet build may look effective on a handpicked set of genuinely stuck tasks. Including healthy quiet work reveals its false interventions. A rule that always chooses a more expensive model may improve completion while wasting owner time on unnecessary review and integration. Outcome and cost have to travel together.

Promotion should bind the exact rule digest, allowed action, evaluation version, approval route and rollback trigger. Configuration reload should retain the last known good version if validation fails. A protective rule needs an expiry or review date so an emergency restriction does not become a permanent silent outage.

The design material describes a broader learning loop across domains. I treat that as a design to qualify, not evidence that an autonomous proposer has safely learned and activated policy in production. A model suggesting a rule, a reviewer accepting its rationale, and a controller safely applying it are three separate events.

Grade the supervisor against failures it can create

The useful test set includes accepted delivery followed by sender timeout, queue consumption during promotion, turn rollover before a steer, monitor restart after unknown dispatch, delayed acknowledgement, duplicate notifications and interleaved human messages. It also includes quiet critical tools, stale configuration and a rule that mistakes an approval wait for a hang.

The positive controls matter as much as the refusals. Can a valid intervention still reach the intended turn? Can a healthy long operation finish? Can a legitimate human message remain untouched? Can a newly installed guard permit the ordinary operation it was supposed to preserve?

One report in the same source window illustrates why: the orchestrator said an installed guard probe caught a missing dependency in a sparse checkout, which would have rejected valid searches as well as undesirable ones. That is a source-reported finding; its raw qualification receipt still needs to be joined before I use it as an independently verified result. A source-level policy can be reasonable while its installed runtime fails closed on everything. Installation needs allowed and denied canaries through the actual route.

For supervision, I would report intervention precision, intent-to-acknowledgement delay, useful work recovered, duplicate/lost intent counts, cancelled useful work, reviewer cost and accepted outcomes per owner hour. I would segment those measures by task class and delivery mode. Otherwise a supervisor that constantly talks can look more effective than one that intervenes rarely and correctly.

Start with one overdue intent

The next useful implementation exercise is small: follow one owned overdue intent from its durable creation record to the recipient's acknowledgement and the verified change in system state. Identify the exact active turn and the current external-effect boundary. Test restart, timeout and stale-turn behavior before enabling automatic escalation.

Then add a bounded GPT or Claude review at the decision point where it supplies judgment the monitor cannot. Let the review propose a permitted intervention or a candidate rule. Keep the action controller, approval route and rollback evidence outside the model's ability to redefine them.

That is how I want supervision to earn its place: fewer decisions escaping attention, fewer repeated repair loops, and more accepted work for each hour of my involvement. Another model is useful when it changes that result. The message saying it tried is only the beginning of the evidence.

How this was verified

The eight-hour census streams the original private root log, deduplicates usage by response ID and exports metadata, digests and source pointers. All six numerical usage fields match a separate cumulative-counter-span calculation. Deliberate duplicate and conflicting-record controls exercise the scanner. Root usage is not a child-model inventory, invoice, isolated model-speed benchmark or accepted-outcome count.

Implementation claims come from the versioned check-in producer, rule library, promotion code and configuration. The watchdog capability receipt is pinned to harness commit d8bfb9acea565333fa61f76ea6088e0af65b35fc. The later 22:21 status update is explicitly outside the fixed census window. No live queue, steering, interrupt, rule activation or publication was performed to research this article.