FIELD NOTE / TIKTOK
'AI made us faster' can be a stopwatch trick.
The complete written thought and the evidence behind it. The video edition will follow its public release.
The written argument is here.
This approved TikTok edition is on the journal now. Its video player and original platform link will appear after each public release is verified.
'AI made us faster' can be a stopwatch trick.
Video caption
'AI made us faster' can be a stopwatch trick. So is the engineer's attention. Ask what the customer got. #EricFieldNotes
Full written post / accessibility read
Imagine a fictional team celebrating that the agent wrote a change quickly. The patch then needs another prompt, another test pass, another deploy and an investigation into what else might be wrong. The useful metric starts with the request and ends with accepted behavior in the product.
Log every run, failed test, human correction, review wait, deployment attempt and defect found afterward. If an agent drafts five candidates before one is accepted, recording only the fifth candidate's generation time is bad accounting. The first green demo is not the denominator.
Choose real task classes, predefine acceptance and capture full elapsed and active time. Keep cases that fail or get abandoned in the dataset. METR's 2026 update warned that selection and multiagent time tracking can distort even a careful experiment, so inspect missing cases before claiming a win.
The rule: measure time and cost per accepted outcome, including rework and failure, then change the harness or process where the delay actually occurs. Do this because faster typing is only valuable when it shortens the path to correct work in users' hands.
#EricFieldNotes
Four-beat scene transcript
1. 'AI made us faster' can be a stopwatch trick.
Imagine a fictional team celebrating that the agent wrote a change quickly. The patch then needs another prompt, another test pass, another deploy and an investigation into what else might be wrong. The useful metric starts with the request and ends with accepted behavior in the product.
Visual: Where did the clock stop?
2. A retry is part of the cost.
Log every run, failed test, human correction, review wait, deployment attempt and defect found afterward. If an agent drafts five candidates before one is accepted, recording only the fifth candidate's generation time is bad accounting. The first green demo is not the denominator.
Visual: So is the engineer's attention.
3. Run a fair local comparison.
Choose real task classes, predefine acceptance and capture full elapsed and active time. Keep cases that fail or get abandoned in the dataset. METR's 2026 update warned that selection and multiagent time tracking can distort even a careful experiment, so inspect missing cases before claiming a win.
Visual: Use matched work and an external oracle.
4. Do not sell the first diff as delivery.
The rule: measure time and cost per accepted outcome, including rework and failure, then change the harness or process where the delay actually occurs. Do this because faster typing is only valuable when it shortens the path to correct work in users' hands.
Visual: Ask what the customer got.
Research and claim limits
- DORA State of AI-assisted Software Development 2025 (S152)
- METR early-2025 experienced-developer RCT (S153)
The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.