FIELD NOTE / X
Two models can agree on the wrong state.
The short film, the complete written thought, and the evidence behind it.
The X conversation link will follow its public release.
Two models can agree on the wrong state.
Video caption
Two models can agree on the wrong state. Approval and revocation can both be true events. Observed records and policy ownership settle the state. Narration uses Eric's authorized AI voice clone. #EricFieldNotes
Full written post / accessibility read
Two capable models agree that an old approved flag should become active access in a replacement service. The demo works. But neither model saw the record where approval was revoked later.
In this illustrative migration, approved is a historical event. The later revocation controls current access. A green screen built from the old flag can be perfectly consistent with its own generated tests.
Let the agents propose mappings, but keep representative historical records and expected states with the policy owner. Ask for a prediction before revealing the held-out revocation case.
Compare the persisted entitlement and audit trail with the owner's expected result. Do this because two fluent explanations cannot supply missing business history or authorize a customer outcome.
Narration uses Eric's authorized AI voice clone.
#EricFieldNotes
Four-beat scene transcript
1. Two models can agree on the wrong state.
Two capable models agree that an old approved flag should become active access in a replacement service. The demo works. But neither model saw the record where approval was revoked later.
Visual: Agreement only covers the brief they both received.
2. The missing record decides the case.
In this illustrative migration, approved is a historical event. The later revocation controls current access. A green screen built from the old flag can be perfectly consistent with its own generated tests.
Visual: Approval and revocation can both be true events.
3. Hold the history outside the build.
Let the agents propose mappings, but keep representative historical records and expected states with the policy owner. Ask for a prediction before revealing the held-out revocation case.
Visual: Ask both models to predict the unseen record.
4. Use agreement to widen hypotheses.
Compare the persisted entitlement and audit trail with the owner's expected result. Do this because two fluent explanations cannot supply missing business history or authorize a customer outcome.
Visual: Observed records and policy ownership settle the state.
Research and claim limits
- DORA 2025 State of AI-assisted Software Development (S225)
- GitHub Copilot Agents application card (S226)
The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.