A request ledger has to survive its own extractor

Watch for a capture layer that can switch itself off without telling anyone. If every request you give an agent fleet is recorded by one extractor reading one stream, then any message that stream does not carry, or any message that stalls the extractor, is a request that never existed as far as the system knows. Nothing errors. The ledger just stops growing.

This shape showed up twice: first in an audit of requests, and then in a bug in the fix.

The pattern: rules that block hold, rules that rely on memory drift

I told my main agent session that I did not see the point of the harness work if the rules did not get followed and things did not get tracked. It answered with a two-column table: the rules I had given it, and what had actually happened. The rules that held were the ones that block: the commit message format, the delivery script, and the rule that any ask put to me must cite my real message. The rules that drifted were the ones that depended on an agent remembering. For "track every request," the second column read: 57 messages missed, and 52 requests stalled with nobody on them.

That split is the whole essay in one line. A rule that is enforced at a gate survives a busy week. A rule that lives only in guidance does not.

Why requests get lost

Before building anything, I had an agent rebuild everything I had asked for over roughly a day and a half. It worked from my typed messages, the write-ahead logs, the code host, the ticket tracker and some light live checks. It also had to read 57 queued messages that the original extractor had missed. When I typed while an agent was busy, those messages went into a queue the extractor never read. Decisions and standing rules were counted as items too.

The final count was 236 items:

The reasons for the stalls taught me more than the totals.

Local only. The most common stall was work that existed on one machine and nowhere else. A run of six related items on usage metering and limits had a branch that was tested and reviewed, but it was never pushed, no pull request was opened, and the test database had since been removed. A single-sign-on epic and a delegated-authority feature stalled the same way. Restart notes from a later data-loss event listed ten local-only findings on one laptop: branches with dozens of commits that never reached the remote, and worktrees with uncommitted fixes.

Promises nobody kept. One item's last evidence was a main-agent note saying the owner had finished its turn and the result still needed checking. The item was never mentioned again.

Claims nobody checked. A protection rule for a main branch had been approved and marked as dealt with. A live check showed the branch was not protected. (It was applied later.) In the other direction, a question had been listed as dropped when it had been answered hours earlier.

The agents then built a coverage matrix: every recurring struggle, and whether anything actually enforced a fix. Its header rule was "guidance in an instructions file alone is not coverage." It found 62 struggles: 5 covered, 19 partial, 21 in flight, 16 missing, 1 dropped. Three of the missing ones were the ones that matter here: promises never kept (two separate rows) and the main agent repeating a subagent's claim without verifying it.

The design: events, not a list

The requirement sounded simple. Every request gets an owner and a state. A request leaves the ledger only when it is done, or when I cancel it with proof, and only after 24 hours.

Three design choices carry it, and I still think they are right:

1. Events are the system of record. The snapshot is a fold over the events, and the index is a fold of the snapshot. Every change is an event with a writer identity and a per-writer sequence number, chained by hash, and appended before the snapshot is rewritten. Syncing machines means taking the union of the event files and folding again. Nothing is overwritten.
2. One actor and one log pointer, used everywhere. Every claim records who made it and points at a location, a message id and a line hash.
3. Verified and attested are separate fields on each piece of evidence. An agent saying something is done is attested. The harness re-checking it against a named system of record is verified. Done requires verified evidence, and the schema enforces it. That is the direct answer to "the main agent repeats a claim without checking."

On privacy, no field can hold raw human text. A request is stored as a hash, counts, a short redacted excerpt, and a key into a private store on the machine that captured it. That follows the same idea as trace agent decisions without turning telemetry into a private content store.

What broke: one message could turn capture off

The first code inspection was done by a Sonnet reviewer standing in for the usual review helper, which was unavailable. Its first finding was blocking.

The function that looked for request cues used a regular expression with a starting branch that allowed a run of whitespace after punctuation or a newline. On a long run of newlines, the engine retries from every newline, consumes the rest of the run, then backtracks. That is quadratic. The reviewer measured it:

• 12,000 newlines: 7.3 s
• 20,000 newlines: 20 s
• 40,000 newlines: 81 s

A slow pattern would have been bad enough. What came after it was worse. The hook had its own 6-second timer, but the timer ran on a separate thread, and in Python a thread cannot fire while the regex engine holds the interpreter lock. The only real limit was the agent runtime's 10-second hook kill. When that fired, the transaction never committed, and the timeout path exited without spooling the capture. On each following prompt, the sync step re-read the same bad line from the same offset and stalled again.

The end-to-end reproduction showed it. The hook ran for 20.9 seconds. The ledger gained no events. The next two prompts, both ordinary, each took about 20 seconds and committed nothing. The only trace was timeout lines in a failures file. Warnings were surfaced by the prompt hook, and the prompt hook never finished, so the warning never reached me.

So a large paste, or any message with a long blank run, would have stopped request capture and end-of-turn enforcement for the rest of the session, silently. The design packet's microbenchmark had looked fine, 9.8 ms on 50 KB, because it used ordinary text.

The same inspection found a quieter problem. Events were folded in timestamp order, with wall-clock time first. The reviewer stepped the clock back between two events from one writer. Live state said in progress. Replaying the events said ready. After a clock step, recovery could also re-apply events that had already been applied. A time-sync correction or a virtual machine restore could reopen a closed ask or close an open one, and the 24-hour removal gate trusted that same clock.

What changed

The fixes were small once the problems had names:

• The cue pattern was fixed. The next review measured 60,000 newlines at 0.035 s.
• Folding and catch-up now order by writer and sequence number, which is the hash-chain order. Wall-clock time only breaks ties between writers.
• A hook timeout now spools its capture instead of discarding it.
• Hook captures carry an HMAC attestation from a per-machine key. Captures recovered from the transcript are marked unverified, and asks split out of them are marked inferred.

The next review found the same class of failure somewhere else. An older redaction step was quadratic on one long unbroken token: 64,000 characters with no whitespace took 11.3 s on a laptop. The new excerpt code was the first caller to hand it 64 KB windows, so one minified blob or encoded dump would lose the capture the same way. The fix was to clip the text before redacting, since the excerpt is cut to 160 characters anyway.

That review also showed that the proof required to cancel a request was too weak. A cancel proof has to be bound to the meaning of the request it closes, not to word overlap, or an unrelated message can close a real ask. That path took several more rounds, and the last review in my notes still asked for changes, so I am not claiming the ledger has merged.

What I take from it

Two points go beyond "give long-running work one durable owner and a handoff that carries state." The audit showed owners and handoffs are not enough on their own.

• The record must not depend on one extractor seeing every message. The first extractor missed 57 queued messages. The second could be disabled by a single message. Capture needs a path that survives the extractor failing: spool on timeout, and re-sync from the transcript with the result marked unverified.
• Event order must not come from wall-clock time. Order by sequence within the writer, and use time only to break ties between writers.

And one new item for my catalogue: a slow pattern in a hook fails as a silent disable. Hooks fail open so they do not block work, which is the right default. But fail-open, plus a regex that cannot be interrupted, plus a warning path inside the same stalled hook, adds up to a guard that turns itself off and tells nobody. If you rely on hooks, the companion question is in your hook ran: did the forbidden action stay blocked?

Checklist

• [ ] Every hook pattern is tested on adversarial input (60,000 newlines; 64 KB with no whitespace) with a wall-clock bound, as a failing control.
• [ ] Input is clipped before any regex or redactor in a hook path. A thread timer cannot interrupt a regex in progress.
• [ ] A hook timeout spools what it captured instead of dropping it.
• [ ] Hook failures surface from somewhere other than the failing hook: a separate end-of-turn check, or a freshness alarm on the ledger.
• [ ] Event logs order by per-writer sequence; recovery refuses, or warns loudly, when the clock has gone backwards.
• [ ] Every input queue (including messages typed while an agent is busy) is read by capture, and a re-sync from the transcript exists.
• [ ] A branch that is not on the remote counts as stalled, and a sweep looks for it.
• [ ] A done claim needs evidence the harness re-checked itself, not an agent's word.

Go deeper on building enforcement that survives a busy week in the Transition2 course.