Build a certainty ladder for agent work.

Video caption

Build a certainty ladder for agent work.

The costly failure sits in execution and recovery.

Use independent evidence for the consequential ones.

My rule: Make absent evidence unverified, not green.

#EricFieldNotes

Full written post / accessibility read

Leaders often ask if a system is reliable, then get one green badge. I want five separate claims: output parses; the value matches live state; the actor is allowed to do it; the side effect happened once; recovery works after interruption.

An illustrative agent approves a supplier payment and renders a valid record. The request later times out after the bank accepts it. A retry can produce a second attempt unless the transaction and reconciliation contract is explicit. Grammar never sees that external state.

The model team owns parse and semantic evaluation. The application team owns policy and authorization. The payment integration owns stable intent and ledger reconciliation. CI injects an ambiguous acknowledgment and reads the provider outcome. The release owner signs the observed result.

A content tag may need syntax and human review. A payment needs authority, exactly-once intent handling and recovery proof. Define the required rung per action and block when the evidence is missing. Do this because a system cannot inherit certainty from a lower layer.

#EricFieldNotes

Four-beat scene transcript

1. Build a certainty ladder for agent work.

Leaders often ask if a system is reliable, then get one green badge. I want five separate claims: output parses; the value matches live state; the actor is allowed to do it; the side effect happened once; recovery works after interruption.

Visual: Each rung answers a different question.

2. A clean demo tests the bottom rung.

An illustrative agent approves a supplier payment and renders a valid record. The request later times out after the bank accepts it. A retry can produce a second attempt unless the transaction and reconciliation contract is explicit. Grammar never sees that external state.

Visual: The costly failure sits in execution and recovery.

3. Assign an owner and test to each rung.

The model team owns parse and semantic evaluation. The application team owns policy and authorization. The payment integration owns stable intent and ledger reconciliation. CI injects an ambiguous acknowledgment and reads the provider outcome. The release owner signs the observed result.

Visual: Use independent evidence for the consequential ones.

4. Gate release on the highest rung needed.

A content tag may need syntax and human review. A payment needs authority, exactly-once intent handling and recovery proof. Define the required rung per action and block when the evidence is missing. Do this because a system cannot inherit certainty from a lower layer.

Visual: Make absent evidence unverified, not green.

Research and claim limits

• OpenAI structured outputs explanation (S08)
• llama.cpp grammar documentation (S09)
• llama.cpp server documentation (S10)

The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.