Agent fleet field notes / Field essay 77

A path contract has more readers than the review names.

When a fix changes a shared path, review finds one broken reader per round. Make the reader inventory a failing test with an empty allowlist before the first fix.

Published

A path contract has more readers than the review names

The pattern to watch for

A review fails a change with one blocker: a reader of some shared value is still doing it the old way. The fix is one line. You make it, send the change back, and the next review fails it again, naming a different reader. Then another.

When that loop shows up, the review is not finding new bugs. It is doing your inventory for you, one item per round, at the overhead of a full review cycle each time.

A task in my agent runner went through four review rounds this way. Each round fixed the reader that review had named, and the next round found one more: a config path, a usage reader, a symlink edge on the stable copy, then a second author path. When a rescue agent at the next rung up read the source instead of the review, it found that the reader named in the latest blocker was one of six still broken. Three of the other five failed silently. The agent downstream would have seen an empty string, not an error.

My rule: when a fix changes a contract, list every reader of that contract mechanically before the first fix, not just the one the reviewer names.

How a stored path became a contract

My runner is the Go program that takes queued work items and hands them to coding agents. Each queued item stores a path to its brief, the markdown file that tells an agent what to build. Hand-queued briefs live in a ready or held folder, keyed by repo and id.

A separate tidy-up tool moves those briefs to a stable location. It does this routinely. The move did not update the path stored on the item. After a tidy-up, the item pointed at a file that no longer existed.

The runner then did one of two things, depending on which code path read the brief. Some paths failed the item with "task brief unavailable". Others read an empty string and carried on, so an agent received a prompt with an empty task contract. Either way, an item whose work was fine ended up failed or wrong.

This had been happening for a while before anyone named it. A coordinator note eventually tied it to repeated flapping that had been holding up a fleet landing gate. Before that, the visible fix was manual: put one brief back at its stored path. That restored one brief and left the cause in place.

"Where is this item's brief" sounds like a lookup. It is a contract, and a contract has consumers.

The fallback was the easy part

The first design idea had two halves: readers fall back to the stable location when the stored path is gone, and the queue tool rewrites the stored path. What was actually built was narrower and stricter. It rewrote no stored state. The rules were:

  • An existing stored path always wins. The fallback is never consulted if the original file is there.
  • Only a missing file is redirected, and only for the exact ready or held shape: no parent-directory segments, the exact file name.
  • The stable file must pass checks. It is contained in the tasks folder (checked through symlink resolution plus a local-path check), it is itself a regular file and not a link, and its own ticket reference matches the item's ticket.
  • One log line per task per process, so the redirect is visible without flooding the log.
  • The queue update command keeps the stable copy current in the same commit as the ready or held copy, and refuses with exit 3 if the stable copy carries a different ticket.
  • Shipping is gated. The runner pins this input, so it ships only after a blind security review, and only in a gap between runs.

The rule I care most about is the first one. A fallback that prefers a second location can be used to swap one brief for another. This fallback only fills a hole, and only with a file that proves it belongs to the same ticket.

All of that lived in one shared helper. Then the readers were converted one at a time: the finish gate, a usage reader, routing policy, failure routing and two calls in the first author path. The helper was right from early on. The gaps were in the list of who called something else.

Why the rounds kept finding one more

Each round's scope was "convert the readers we know about". The review then found the next one. Reviewers report what they find. They do not promise that what they found is everything. Fixing only the named reader treats the review as the inventory, and a review is not built to be one.

Here is what the source read turned up, with the failure behavior chosen for each:

Reader What it did after a move Behavior chosen
Second author path, prompt prefix Read empty on any error, then cached the empty prefix, so the empty contract survived retries Use the shared helper. On failure, write no cache and fail the item as "task brief unavailable"
Replay plan Built its file list from a raw read Reuse the text the author path already resolved
Merge repair Child brief embedded an empty "original brief" Embed the resolved brief, or an explicit "Original brief unavailable" marker; return an error only where every caller handles it
Rescue context (two sections) Silently empty Write "brief unavailable" with the error into the section; never abort the rescue over it
Integrate select Failed closed, which blocked integration Resolve the path, run the same checks on the resolved bytes, record the path actually read; an unresolved brief still errors
A file scanner Silently skipped a missing file Scan the resolved path when there is one

The dangerous ones share a trait. They turned an error into an empty string. The worst of them did it behind a cache, so even after the stable copy turned up, retries kept serving the empty contract. A reader that fails loudly loses one item. A reader that fails quietly hands an agent a blank brief and lets it build something.

How to detect it: make the inventory a test

The rescope kept every fallback rule exactly as it was and added one control that I now reach for whenever a contract changes.

It is an inventory test. It parses every non-test Go file in the runner with Go's standard parser and syntax-tree packages. It fails on any file read, open or stat call whose argument is the item's brief path field, anywhere outside the one helper file. The allowlist starts empty. Each entry needs a file, a function and a written reason. If the pattern catches a field that is not really a path, that field goes on the allowlist with its reason. It is not filtered out quietly.

Two details make it trustworthy.

First, it had to show red against a mutant before it showed green. The mutant puts a raw read back into the second author path, and the test has to name the file and line. A test that has never failed has not shown that it can.

Second, the brief asked the engineer to list every finding the test made before any fixes. So the inventory itself is on record, not just the final green run. That list is what a reviewer can check against the source.

An AST check with an empty allowlist is a regression test that also covers readers that do not exist yet. The next person who adds a raw read of the brief path gets a failure with a file and line, not another review round.

This is the same instinct behind building the test harness around cross-system invariants: the invariant is "every reader resolves through one helper", and the test states it in code.

How to prevent it

Detection finds the readers. Prevention is about what each reader does when the contract cannot be met.

Write failure behavior per reader. The table above has four different answers: fail the item, write an explicit marker, keep today's error, or scan what exists. Each fits its caller. What none of them allows is "fall back to empty", and that should not be an option anyone can pick by accident.

Keep the fallback narrow, and write down what it does not do. This one does not rewrite stored state. It does not read the stable copy when the stored path exists. It does not help a brief that was never copied to the stable location. Saying that plainly stops the next change from quietly widening it.

Treat a blocker as one instance of a class. When a review names a reader, I assume there are more and search for the class before I touch the named one. That is the same reason I want an independent exam to probe the class rather than the instance.

Check that the fix is on main. A good rescope brief is a design, not a deployment. When I looked for this one, the branch carrying the fallback work was not an ancestor of main, main did not contain the helper, and the tidy-up tool was still moving briefs. I have not traced whether the continuation landed another way, so what I can describe is the design and the rescope, not a shipped fix. Before I call a contract change done, I check main for the helper and the inventory test themselves, not the branch.

The short version

The fallback was never the bug. The bug was that "where is this brief" had more consumers than anyone had listed, and each round of review found one more. Once the list is code, review goes back to judging the change instead of discovering its scope.

Checklist

Copy this into the brief for any change that alters a shared path, field or lookup:

  • [ ] Name the contract in one sentence ("every reader of X resolves through helper Y").
  • [ ] Write the inventory test first: parse the source, fail on any direct use outside the helper, start with an empty allowlist.
  • [ ] Record every finding the test makes before fixing anything.
  • [ ] Give each allowlist entry a file, a function and a written reason.
  • [ ] Show the test red against a mutant that restores one raw reader, and confirm it names the file and line.
  • [ ] Write failure behavior per reader: fail, marker, existing error or partial scan. No silent empty string.
  • [ ] Look for caches that could persist a bad read across retries.
  • [ ] Keep the fallback narrow: existing path wins, exact shapes only, identity must match.
  • [ ] State in writing what the fallback does not do.
  • [ ] When review names one reader, search for the class before fixing the instance.
  • [ ] Before calling it done, confirm the helper and the test are on main.

Go deeper: the contract and verification habits behind this are part of the Transition2 course.

CONNECTED READINGForward and back
Keep reading

The answer deserves
another experiment.

Back to the journal