A good engineer tries to break the green test.

Video caption

A good engineer tries to break the green test.

Name, perturb, revise.

It can expose a weak gate without a production incident.

Do not moralize it.

#EricFieldNotes

Full written post / accessibility read

When an agent builds a feature and writes the tests, its code and tests can share the same mistaken premise. The discipline I want is visible: define the expected outcome outside the agent, introduce a meaningful counterexample and see whether the gate catches it.

In a short interview, ask three things: what product result must hold, what counterexample would test it, and how would a failed result change release? An older queued event, lagging consumer or revoked permission gives the interviewer a concrete prompt. Record the reasoning against prewritten anchors; do not assign the fixture as unpaid homework.

Suppose the interviewer reveals that an old event cannot be read by a lagging consumer, although the producer suite is green. A strong answer names the mismatch, asks the compatibility owner and alters the rollout contract or pauses release. It does not merely ask the agent for another producer test. The interviewer supplied the failure evidence.

Test whether a candidate seeks a falsifying fact after the first plausible answer, and whether they can explain the system well enough to act on it. Do this because a green result from the same generation loop is weaker than an independent business outcome. Calibrate the rubric before using it for real decisions.

#EricFieldNotes

Four-beat scene transcript

1. A good engineer tries to break the green test.

When an agent builds a feature and writes the tests, its code and tests can share the same mistaken premise. The discipline I want is visible: define the expected outcome outside the agent, introduce a meaningful counterexample and see whether the gate catches it.

Visual: They know what the test assumes.

2. Observe three actions, not a personality adjective.

In a short interview, ask three things: what product result must hold, what counterexample would test it, and how would a failed result change release? An older queued event, lagging consumer or revoked permission gives the interviewer a concrete prompt. Record the reasoning against prewritten anchors; do not assign the fixture as unpaid homework.

Visual: Name, perturb, revise.

3. A red result is useful evidence.

Suppose the interviewer reveals that an old event cannot be read by a lagging consumer, although the producer suite is green. A strong answer names the mismatch, asks the compatibility owner and alters the rollout contract or pauses release. It does not merely ask the agent for another producer test. The interviewer supplied the failure evidence.

Visual: It can expose a weak gate without a production incident.

4. Make verification a hiring observation.

Test whether a candidate seeks a falsifying fact after the first plausible answer, and whether they can explain the system well enough to act on it. Do this because a green result from the same generation loop is weaker than an independent business outcome. Calibrate the rubric before using it for real decisions.

Visual: Do not moralize it.

Research and claim limits

• U.S. OPM: Work Samples and Simulations (S151)
• U.S. OPM: Designing an Assessment Strategy (S156)

The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.