AI engineering ROI starts at the customer promise.

Video caption

AI engineering ROI starts at the customer promise.

Intent, build and operation have separate owners.

Speed is valuable only after acceptance.

My rule: The gate defines what working means.

#EricFieldNotes

Full written post / accessibility read

I would not measure an agentic team by commits or passing tests alone. In a fictional enterprise migration, three agents faithfully implement a shortened instruction that widens a one-tenant exception. The output is high. Product alignment is low because the source agreement was lost.

Record the source customer decision and exception owner; bind independent allowed-and-denied tests to the exact artifact; then inspect a bounded deployed cohort and rollback signal. A green badge on one rail cannot silently stand in for a missing rail. Each can be pass, fail or unverified.

For every release, track whether the original product decision survived, how many re-prompt and QA loops were needed, who overrode an unknown, whether the canary reached the right cohort and what customer behavior changed. Include incident correction and consciously stopped work in the ledger.

My rule: authorize the exact build only when source intent, independent behavioral evidence and bounded operation agree; expose unresolved exceptions to an owner. Do this because an agent team can accelerate implementation while losing product truth at the handoff.

#EricFieldNotes

Four-beat scene transcript

1. AI engineering ROI starts at the customer promise.

I would not measure an agentic team by commits or passing tests alone. In a fictional enterprise migration, three agents faithfully implement a shortened instruction that widens a one-tenant exception. The output is high. Product alignment is low because the source agreement was lost.

Visual: Generated throughput can increase wrong work.

2. Keep three evidence rails.

Record the source customer decision and exception owner; bind independent allowed-and-denied tests to the exact artifact; then inspect a bounded deployed cohort and rollback signal. A green badge on one rail cannot silently stand in for a missing rail. Each can be pass, fail or unverified.

Visual: Intent, build and operation have separate owners.

3. Count the complete outcome and rework.

For every release, track whether the original product decision survived, how many re-prompt and QA loops were needed, who overrode an unknown, whether the canary reached the right cohort and what customer behavior changed. Include incident correction and consciously stopped work in the ledger.

Visual: Speed is valuable only after acceptance.

4. Buy accepted value, not generated lines.

My rule: authorize the exact build only when source intent, independent behavioral evidence and bounded operation agree; expose unresolved exceptions to an owner. Do this because an agent team can accelerate implementation while losing product truth at the handoff.

Visual: The gate defines what working means.

Research and claim limits

• Google SRE Workbook: Canarying Releases (S149)
• NIST SP 800-218 Secure Software Development Framework (S150)
• DORA State of AI-assisted Software Development 2025 (S152)

The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.