FIELD NOTE / X
The useful engineer keeps going after green.
The complete written thought and the evidence behind it. The video edition will follow its public release.
The written argument is here.
This approved X edition is on the journal now. Its video player and original platform link will appear after each public release is verified.
The useful engineer keeps going after green.
Video caption
The useful engineer keeps going after green. The test and patch share one assumption. Curiosity must change the release verdict. #EricFieldNotes
Full written post / accessibility read
Call it anti-laziness if you want, but I would score behavior instead of personality. When an agent reports a fix, does the engineer ask what result would prove it wrong? Do they find an independent oracle and test a boundary the agent did not choose for itself?
Present a fictional producer patch and tests that assume all consumers upgrade together. Ask the candidate what those tests cannot establish. A strong answer identifies replayed old events, a lagging consumer and the independent business result needed before release. This takes a focused conversation, not a multi-day build.
Ask what old event and expected business result the team should put into a consumer replay. Then ask which deliberate version change should turn the gate red. If it would stay green, the test does not protect the claim. Listen for whether that finding changes the candidate's release recommendation; the team runs the fixture afterward.
My rule: reward the candidate who names the uncertainty, specifies the smallest falsifying test and changes course when its hypothetical result fails. Do this because generated output can look convincing while the underlying contract remains unknown. Check technical fundamentals in the same bounded interview so good questions are backed by engineering competence.
#EricFieldNotes
Four-beat scene transcript
1. The useful engineer keeps going after green.
Call it anti-laziness if you want, but I would score behavior instead of personality. When an agent reports a fix, does the engineer ask what result would prove it wrong? Do they find an independent oracle and test a boundary the agent did not choose for itself?
Visual: A plausible result is the start of review.
2. Show a false green in the interview.
Present a fictional producer patch and tests that assume all consumers upgrade together. Ask the candidate what those tests cannot establish. A strong answer identifies replayed old events, a lagging consumer and the independent business result needed before release. This takes a focused conversation, not a multi-day build.
Visual: The test and patch share one assumption.
3. Ask how the test could fail on purpose.
Ask what old event and expected business result the team should put into a consumer replay. Then ask which deliberate version change should turn the gate red. If it would stay green, the test does not protect the claim. Listen for whether that finding changes the candidate's release recommendation; the team runs the fixture afterward.
Visual: A negative control checks the gate.
4. Score the next discriminating action.
My rule: reward the candidate who names the uncertainty, specifies the smallest falsifying test and changes course when its hypothetical result fails. Do this because generated output can look convincing while the underlying contract remains unknown. Check technical fundamentals in the same bounded interview so good questions are backed by engineering competence.
Visual: Curiosity must change the release verdict.
Research and claim limits
- U.S. OPM: Work Samples and Simulations (S151)
- METR early-2025 experienced-developer RCT (S153)
- U.S. OPM structured interviews (S203)
The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.