FIELD NOTE / TIKTOK
Both models approved the backfill.
The short film, the complete written thought, and the evidence behind it.
Both models approved the backfill.
Video caption
Both models approved the backfill. Replica lag crosses the product's freshness limit. Use peak-like load and a product threshold. #EricFieldNotes
Full written post / accessible read
In this simulated review, Astra proposes a batched database backfill. Fable challenges it, then agrees. The plan sounds responsible. A peak-like rehearsal says otherwise.
Both models saw one quiet hour, not the peak tenant's write skew. Under the missed workload, replica lag crosses the product's freshness requirement. A customer acts on stale data.
The rehearsal trace is simulated, but the engineering distinction is real: a debate over the same incomplete sample cannot discover the omitted workload. The test has to supply peak-like input and an explicit lag threshold.
Before rollout, run a peak-like rehearsal, compare observed lag with the product's freshness limit, and assign rollback ownership. Do this because model agreement is only a hypothesis until a workload can break it.
#EricFieldNotes
Evidence and boundary
On-screen label: SIMULATED REHEARSAL. Illustrative cases are not measured incidents. Research papers and vendor documents support the stated mechanism only within their studied or documented scope.
- METR randomized trial (S20)
- SWE-bench Live paper (S22)