Why agents cannot grade their own work
Why can't the agent that wrote the change also tell me the change is done? Because "done" is a claim, and a claim from the author is not evidence. I want code, not the author's confidence, deciding whether work counts.
This is the first of five essays about a small open-source agent harness I maintain, Agent Harness. I pin everything here to one commit, 504d930, so the commands and output below are reproducible. The harness is instructions plus small Python helpers. Its own README says it is not a daemon, not a CI service, and not a guarantee that an agent follows every step. I will keep that framing: these are controls that make the careless path hard and the honest path easy. They are not a security boundary against an agent that is trying to get around them.
A prompt is a request, not a rule
Picture an agent asked to tidy a project folder. It tidies, then decides the old session logs are clutter and deletes them. Nobody told it not to. The model did not malfunction; it did the helpful thing it could think of, and now the only record of what happened is gone.
You can tell the model not to delete logs, and you should. But the model reads that instruction, weighs it against everything else in front of it, and usually follows it. Usually. A harness is the code around the model that turns a request into a rule: the model proposes an action, and the harness decides whether it runs.
The smallest version has four parts: a loop that asks for the next action, a list of tools, a check between the proposal and the tool, and a record of what happened. I wrote that toy on a fake machine, a Python dictionary, so nothing real is at risk. Its check is ordinary code:
def policy(argv):
"""Code decides, not the model. Returns None to allow, or a reason to block."""
cmd, *args = argv
if cmd == "cat" and any(a in SECRET_PATHS for a in args):
return "reading a credential file can leak secrets"
if cmd == "rm" and any(a.startswith(PROTECTED_PREFIXES) for a in args):
return "deleting a log removes the only record"
return None
Plain code gives the same answer every time, costs nothing and can be tested. A model can help decide what the rules should be; it should not be what enforces them.
The toy has four tests. One is a control with the guard switched off: the credential file prints and the log disappears from the dictionary. That is the failure the guard exists to prevent. Then I broke the delete rule on purpose and the "log must survive" test failed, which is the only way I know that the test can see the problem:
$ sed 's/ if cmd == "rm" and any/ if False and any/' harness.py > /tmp/hx01-mut/harness.py
$ cd /tmp/hx01-mut && python3 -m unittest test_harness
AssertionError: False is not true
FAILED (failures=1)
With the rule restored, python3 -W error -m unittest -v ran 4 tests and ended OK. One of the four documents a limit of the toy rather than a feature: a path with a stray /./ in it slips past the prefix check, because the toy compares strings, not meanings.
The real guard, same shape
The real command guard in the harness has far more cases behind the same idea. check prints a decision and runs nothing. With a throwaway home directory:
$ command_guard.py check "ls -la"
{"verdict": "allow", "message": ""}
exit 0
$ command_guard.py check "cat ~/.aws/credentials"
{"verdict": "block", "message": "command_guard BLOCKED [secret_dump/credential-file]: `cat` of a credential file can print or send secrets (credentials). Safe alternative: check existence with `test -s FILE` or `ls -l FILE`; use the owning tool's own config flag; to count entries use `grep -c`. There is no override variable; ask the user if this command is really needed."}
exit 2
Deleting the agent's project logs and printing the whole environment were blocked the same way, each naming the rule and a safer alternative. I will say more about guards in the fourth essay. For now the point is only the shape: a proposal goes in, a decision comes out, and the decision is not a matter of opinion.
The same idea, applied to a reviewer's approval
A guard stops one dangerous action. The harder problem is the end of the work. An author who says "the tests pass" is reporting on their own work, with their own blind spots, and nothing about that sentence can be checked. So the harness asks for a review by someone else, and then it checks the review, because a review can be real and still describe code that no longer exists. Picture a branch with a green review attached. The author pushes three more commits, then merges, pointing at that green. The reviewer never saw those commits.
The harness has three review stages, and they ask different questions at different prices:
- plan asks whether this is the right design, before anyone writes it. A wrong plan is cheapest to fix here.
- code-inspection asks whether the code is right, with full validation explicitly still pending.
- pr asks whether the evidence at the final commit supports merging. Only a passing
prresult counts as sign-off.
The review helper writes a header naming the stage, the commit it reviewed and a SHA-256 of the packet it read. The packet is the evidence file handed to the reviewer: scope, commands, results. The hash covers the raw bytes, so one changed character changes it. A consumer compares all three to what it holds now, and then looks at the first line of the body for the marker that belongs to that stage.
Nothing about that is clever, which is the point: plain checks are testable. My toy gate is 61 lines, and the three comparisons that matter are these:
if fields.get("Stage") != stage: # MUTANT-STAGE
return False, "wrong stage"
if fields.get("Reviewed HEAD") != commit: # MUTANT-COMMIT
return False, "stale commit"
if fields.get("Reviewed packet SHA-256") != sha256(packet): # MUTANT-PACKET
return False, "packet changed"
Eight tests pass. Then I planted three bugs, each removing one comparison, and the suite had to fail every time:
mutant COMMIT: FAILED (failures=1)
mutant PACKET: FAILED (failures=1)
mutant STAGE: FAILED (failures=1)
A test that cannot catch a planted bug will not catch an unplanted one. The stale-commit mutant is the opening scene: a gate that accepts an approval for code that has moved on.
What the real helper does, offline
The real review helper is full of model calls, and I did not run any. What I ran are its offline parts, against a scratch git repository with a throwaway home folder. A code-inspection request with no expected commit is refused before anything launches. A packet that only says "tests pass", with no command and no exit status, fails lint:
== 1. code-inspection with no expected commit
exit 2
peer_review.py: error: --expected-head is required for code-inspection and PR review
== 2. code-inspection, packet says tests pass but shows no command or exit status
exit 2
Packet lint failed for the code-inspection stage; no provider was launched.
Then I called the report-format functions directly. A plan approval offered as a pull-request approval is rejected on the stage and the marker:
verdict_match(pr, 'PLAN_RESULT: ACCEPT') -> False
verdict_match(plan, 'VERDICT: PASS') -> False
verdict_match(plan, 'PLAN_RESULT: ACCEPT') -> True
CODE_INSPECTION_RESULT: ACCEPT
[PR signoff marker omitted] in the body
stage_report(pr, plan marker) raises OSError: reviewer verdict does not match the requested stage
The fourth and fifth lines show a detail I like: when a code-inspection report quotes the exact pass marker in its body, the helper rewrites that string so an inspection can never be mistaken for sign-off. It matches only that exact string, so the stage field is what really decides, and a person should still read the report.
What a matching hash does not prove
A matching hash proves which bytes and which commit the reviewer read. It proves nothing about the reviewer's judgment. It is also not race-proof: the harness documentation describes a small window in which a packet can change between the check and the report. And the packet lint is a format guard; it cannot prove a run happened, only that the packet shows a command with an exit status. If you edit the packet after a review, the old report is dead for that packet, by design. Rerun the stage, using a fresh report path.
The workflow this supports is simple. Get code inspection early and cheaply, with the full suite pending. Repair what it finds. Freeze the code, run full validation once, then ask for the final review at the exact final commit. The harness guide, HARNESS.md, describes that sequence under "Plan, review, and sign-off".
What I did not run
- No model review ran. A real plan, inspection or PR review needs a provider CLI and paid access. Everything above is the offline half, and the toy reviewer in my gate is a stand-in that writes the same header.
- The guard is best-effort. It catches the command shapes it knows. My toy visibly misses a dotted path, and a known gap in the real guard has been reported; I am not describing it here.
- Commands are shown as run on a scratch repo with a throwaway home. Hashes, timestamps and ids vary between runs, so the lab masks them when it compares output.
Read the last approval you relied on and find which commit it covers. The next essay is about what an approval is attached to, and what a spec has to say before anyone reviews anything: The spec, the reviewers and the evidence ledger.