FIELD NOTE / LINKEDIN
A model loop won't make you an engineer.
The short film, the complete written thought, and the evidence behind it.
The LinkedIn edition will be linked here after its public post is verified.
A model loop won't make you an engineer.
Video caption
A model loop won't make you an engineer.
The loop cannot see an unwritten customer limit.
Topology, failure history and customer tolerance matter.
What I would require: Evidence, residual risk, rollback condition.
#EricFieldNotes
Full written post / accessible read
Running Astra and Fable against each other can improve a design. It does not make you an engineer, and it does not make their agreement an acceptance test.
After a bad migration, a customer cannot ask which model forgot the recovery requirement. Somebody in your team owns that failure. The loop cannot measure a workload it never saw or infer an unstated product promise.
Take a proposed architecture and ask what observation would make it wrong. Bring in actual topology, failure history and customer limits. Rehearse the riskiest boundary before approving the design.
Use the loop to widen the options. Let measurements narrow them. Then have an engineer record the choice, remaining risk and rollback trigger. Do this because a persuasive answer cannot absorb the consequence of a wrong one.
#EricFieldNotes
Evidence and boundary
On-screen label: ERIC'S POSITION. Illustrative cases are not measured incidents. Research papers and vendor documents support the stated mechanism only within their studied or documented scope.
- METR randomized trial (S20)
- SWE-bench Live paper (S22)