A working journal by Eric MacDougall
Notes from inside the work.
On agents, engineering judgment, infrastructure, and the gap between a convincing demo and a system you can trust.
Work together Projects, fractional leadership and advisory.
Built in public, with sources and the edges left visible.
better evidence.FIELD NOTE 01
01 / Watch
The films, in one playlist.
64 overview films. Each one has a full written essay behind it.
02 / Latest essays
Long reads, newest first.
Late replies must match the current identity before they write
Four products, one bug: a late reply, cleanup step or expired lease changed current state. Carry identity into every write and test the orderings with a controlled clock.
13 min read · Published Nov 5, 2026A path contract has more readers than the review names
When a fix changes a shared path, review finds one broken reader per round. Make the reader inventory a failing test with an empty allowlist before the first fix.
12 min read · Published Nov 5, 2026A request ledger has to survive its own extractor
An audit of 236 requests found 52 stalled, and the capture fix could be silenced by one long message. How to build request capture that cannot switch itself off.
12 min read · Published Nov 5, 2026Parallel branches need one owner for the migration index
Agent branches that pick their own migration numbers collide. A landing order with one owner, reserve-first indexes and a merge-based re-chain recipe keep the chain intact.
11 min read · Published Nov 5, 2026Nothing that runs should live where cleanup can reach it
An agent harness that grew in a scratch folder lost most of its runnable code in about 80 seconds. How to spot the pattern, recover, and keep code and gates out of cleanup's reach.
12 min read · Published Nov 5, 2026When two hooks disagree, write the precedence order down
A Stop hook said push before stopping and the runner said never push. Two agents resolved it correctly, and a written precedence order keeps that from resting on judgment.
10 min read · Published Nov 5, 2026The agents used 8% of the host. The polling around them used more
Agent engines took 3.6 of 44 cores while supervisors forked 5,751 processes a second, and the idle host was disk-bound on shared storage. Measure the control plane first.
12 min read · Published Nov 5, 2026Check that each git worktree resolves to itself
One shared git setting made 17 linked worktrees write into a live tree. How to detect it with one command, prove the fix with a negative control and keep worktree isolation real.
11 min read · Published Nov 5, 2026A fail-closed allow-list can silently disable your failover
A strict model allow-list refused every request on a relay's backup path for two weeks. How to spot it, probe fallbacks with real payloads and alert on zero successes.
11 min read · Published Nov 5, 2026Fourteen control files became one release command
A site release had 14 control files and a 923 MB archive, and still could not ship. What one refusing, reading-back release script kept, and the gap it still has.
11 min read · Published Nov 5, 2026Measure the split before you redesign the agent loop
A five-second computer-use click had about 0.3 seconds of server time. Timing each verb two ways killed a push-channel redesign and found the real overhead in the helper path.
12 min read · Published Nov 5, 2026Tests that spawn processes need a group, a cap and a stop order
A loop test without its own cap left about 49,000 orphaned processes. How to contain spawning tests with process groups, an independent cap, and a cleanup order that ends the growth first.
11 min read · Published Nov 5, 2026Landed is not done: define done before agents report it
A status audit found seven false completion claims, including "proven live" beside 217 failed runs of 246. How to define done so only a live check can satisfy it.
11 min read · Published Nov 5, 2026A Rate Is a Belief With a Width: Updating From Counts to Live Traffic
Pseudo counts make a rate's uncertainty explicit, pooling steadies thin slices, and probability matching routes traffic while you learn. Worked in counts.
16 min read · Published Nov 5, 2026Counting Honestly: Ranges, Zero Failures and Paired Changes
Report every AI success rate as a range, gate automation on its lower bound, size clean test sets from the bar, and judge a change by what it flipped.
15 min read · Published Nov 5, 2026Your answer key is the product: building a gold set you can trust
A model is only as measurable as its gold set: blind, double labelled, stratified, versioned, split by time, and checked with kappa before you trust it.
15 min read · Published Nov 5, 2026Pick the metric by what a mistake does next
Precision, recall and miss rate are not a matter of taste. The harm of each wrong action picks the metric, and walking the threshold sets its value.
12 min read · Published Nov 5, 2026Counts out of a thousand: what a flag really means
A 90% detector can be wrong about most of what it flags. Count out of 1,000 at the real base rate before you let a flag block, note or route.
12 min read · Published Nov 5, 2026A 99% approval rate is not an accuracy number
A reviewer approved 99% of drafts while 1 in 12 was wrong. How to measure correctness with a blind audit, outcome truth and seeded items.
13 min read · Published Nov 5, 2026▶ FILMLet a business operator request a site change without handing AI a deploy token
A non-engineer can request a bounded change without becoming a proxy for unchecked code. Run generation without network or deployment credentials, enforce an exact file boundary, build and review a pinned artifact, and promote through a separate trusted deploy authority.
6 min read · Published Oct 22, 2026▶ FILMThe offline test that spent the production quota
A fake transport is not a network boundary, and an empty negative control is not evidence. Deny egress at the test runner, keep production credentials out, prove the block with a harmless control, and preserve an unknown outcome when readback is unavailable.
6 min read · Published Oct 22, 2026▶ FILMThe 192-core server was still waiting on one kernel worker
High CPU capacity does not guarantee progress when a serialized cleanup path stalls. Reduce sandbox churn, terminate whole process groups, add mechanism-specific early signals, and replay alarms against observed samples before retuning. Keep a local diagnosis distinct from an unverified upstream cause.
6 min read · Published Oct 22, 2026▶ FILMLet an agent ask for a credential without ever seeing it
Separate the agent’s request from the secret value. Bind operator input to the live run, requesting step, target and logical credential; resume the waiting work without returning the value to the transcript.
6 min read · Published Oct 22, 2026▶ FILMShared Git state is the fleet coordination bug
Git worktrees isolate checked-out files but leave shared repository metadata and remote publication requiring coordination. Several observed failures had distinct causes and required different controls.
6 min read · Published Oct 22, 2026▶ FILMRestart is where a deletion tool can lose the only backup
A deterministic archive key and a completion receipt written after source deletion can make a retry overwrite the only backup. Model each external write, verification and destructive local effect as durable states.
6 min read · Published Oct 22, 2026▶ FILMWhen an agent writes a one-off script, it skips your safety rails
A one-off operations script can bypass approval capture and host refusal even when the underlying operation works. Make the shared CLI the policy boundary and improve it when agents reveal missing options.
6 min read · Published Oct 22, 2026▶ FILMWhat the subscription meter actually charged for tokens
API list prices were a poor proxy for subscription quota. Isolated token-class experiments measured reads, writes and output separately and changed the optimization ranking.
6 min read · Published Oct 22, 2026▶ FILMThe five-minute cache cliff that made waiting subagents expensive
A short cache lifetime mismatched to subagents waiting behind long commands caused large prompt-prefix rewrites. Configure lifetime by worker role and verify positive and negative paths; meter savings remain unmeasured.
6 min read · Published Oct 22, 2026▶ FILMOne per-session line broke the shared prompt cache
A per-session value in an early tool description made otherwise stable headless prompts diverge; route dynamic values only to the client that consumes them and keep remaining entropy separate.
6 min read · Published Oct 22, 2026▶ FILMA merge gate must prove the tests ran
A successful process exit does not prove tests executed; gates must distinguish command success, test execution, baseline comparison and unknown evidence.
6 min read · Published Oct 22, 2026▶ FILMA merge needs proof of no change
A similarity heuristic cannot prove reviewed work is already merged; simulate the exact merge against current main and compare the resulting tree, including reverts relative to the task fork point.
6 min read · Published Oct 22, 2026▶ FILMKey rotation has two readers and one writing authority
During key migration, prepare and try one decryption candidate at a time so malformed fallback configuration cannot break a valid current-key read; new writes still use one current authority.
6 min read · Published Oct 22, 2026▶ FILMRetire worktrees by consumer lifetime, not terminal task status
A terminal task state is only a candidate; safe worktree retirement requires the last consumer to release a shared lease, a fresh owner snapshot and verified private recovery outside the target.
5 min read · Published Oct 22, 2026▶ FILMDesign the evidence-review screen before automating the decision
An evidence-review product must separate read-only inspection, bounded extraction proposals, conflict review and explicit human acceptance; the model does not inherit the decision right.
6 min read · Published Oct 22, 2026▶ FILMA read-only observer still needs backpressure
Read-only describes data mutation, not resource cost. A safe observer checks pressure during long reads, stops only its owned child, preserves last-good data and makes incomplete evidence explicit.
5 min read · Published Oct 22, 2026▶ FILMA fleet pause must reach the process tree
A task is not cancelled until its lifetime reaches admission waits, Git, install descendants and filesystem recovery; cancellation must leave a state the next attempt can trust.
6 min read · Published Oct 22, 2026▶ FILMThe dependency cache belongs before the credential boundary
A dependency cache is shared mutable state. Bind identity to exact inputs, publish complete trees atomically, limit install credentials and choose deliberately between shared reuse and per-step copy.
5 min read · Published Oct 22, 2026▶ FILMMeasure context costs before building an agent steward
Model the fleet-wide share a context change reaches before funding a Steward. Injected-text share, cached-token cost and total usage are different denominators.
5 min read · Published Oct 22, 2026▶ FILMA safe AI widget is a small capability system
Treat model-authored JavaScript as executable content: constrain its language, isolate browser capabilities, pin the trusted bridge and bind the serving route to the reviewed version.
6 min read · Published Oct 22, 2026▶ FILMMake model failover preserve the agent contract
A fallback route changes more than a model name: tool representation, role separation, budgets, privacy and refusal behavior can all differ.
11 min read · Published Oct 20, 2026▶ FILMDesign recovery semantics into the safety state machine
A safety gate needs to preserve evidence and conservative uncertainty while distinguishing historical observations from current authority to hold execution.
11 min read · Published Oct 20, 2026▶ FILMDecompose work around contracts and integration costs
Use the whole-versus-pieces experiment to explain when decomposition creates useful independent boundaries and when repeated setup fragments an interdependent implementation.
12 min read · Published Oct 20, 2026▶ FILMPersonal agents need private authority, recoverable work and measured operating budgets
Long-running personal agents need user-specific compute, storage, credentials and network authority. Multiplexed conversations alone do not create isolation.
11 min read · Published Oct 20, 2026▶ FILMTrace agent decisions without turning telemetry into a private-content store
Keep the links needed to reconstruct decisions and reconcile effects while minimizing stored prompts, credentials and sensitive low-entropy data.
11 min read · Published Oct 20, 2026▶ FILMPort agent controls across runtimes without assuming hook parity
Different CLIs, IDEs and operating systems expose different observation and denial points. Port the contract, then prove each runtime route.
11 min read · Published Oct 20, 2026▶ FILMBuild the test harness around cross-system invariants
A test should encode the business or security promise across components, with an oracle independent of the actor and an outer containment boundary.
12 min read · Published Oct 20, 2026▶ FILMSeparate probabilistic development from deterministic shipping
Models can propose changes and repairs. A narrow release controller can decide whether exact reviewed and tested artifacts meet a fixed contract, without reasoning its way around it.
11 min read · Published Oct 20, 2026▶ FILMRegional cells need one home for mutable authority
Portable containers do not make state ownership, residency and failover portable. Define one authoritative home for mutable state before adding regions.
12 min read · Published Oct 20, 2026▶ FILMAgent fleets need explicit ownership transfers before more workers
More workers help only when task ownership, resource admission and shared-state authority remain coherent.
12 min read · Published Oct 20, 2026▶ FILMParallel AI checks need a release contract before they need more workers
Parallel checks help only when the release condition, ownership of output fields and behavior of late branches are defined in advance.
12 min read · Published Oct 20, 2026▶ FILMMemory success has to confirm the requester, address and receipt
Memory changes future behavior across turns. Tenant scope, namespace, origin, current permissions and deletion semantics matter as much as retrieval relevance.
12 min read · Published Oct 20, 2026▶ FILMA retrieval system needs to know who will read the answer
Requester authorization and answer-audience authorization are separate questions. A user may be allowed to read something that cannot be disclosed into the destination conversation.
11 min read · Published Oct 20, 2026▶ FILMDesign agent supervision around delivery, intervention and bounded learning
Use deterministic event observation, bounded independent model judgment and a constrained intervention controller; spend review effort where it changes accepted outcomes.
13 min read · Published Oct 19, 2026▶ FILMBudget agent work before dispatch, then reconcile what actually happened
A token counter after completion cannot enforce a hard pre-dispatch exposure bound across parallel work and ambiguous provider outcomes.
11 min read · Published Oct 19, 2026▶ FILMRoute AI work by the cost of accepted outcomes
Choose among routes that preserve privacy, authorization, tool semantics and required quality, then optimize expected accepted-work cost under quota and latency constraints.
11 min read · Published Oct 19, 2026▶ FILMTurn engineering standards into agent execution contracts
Agents need explicit constraints at admission, implementation, review and release, connected to the product decision that justified them.
11 min read · Published Oct 19, 2026▶ FILMRecheck current policy authority before protected bytes leave
An immutable signed policy revision is an evidence artifact. Current authorization can change while model inference or transformations are running.
11 min read · Published Oct 19, 2026▶ FILMUnknown is an outcome: designing agent systems around lost replies
A timeout does not determine whether an external action happened. Retrying safely requires a durable operation identity and a reconciliation path.
11 min read · Published Oct 19, 2026▶ FILMDurable agents need agreement between delivery, ownership and effects
A workflow is more than a queue. State changes, durable delivery, worker ownership and publication authority must agree across retries and crashes.
11 min read · Published Oct 19, 2026▶ FILMWhere JEV earned a decision—and where we replaced it
Evaluate decision systems per class and effect: dynamically constrain candidates, use JEV where measured, and replace or escalate it when deterministic scenarios or stronger models own the uncertainty better.
11 min read · Published Oct 19, 2026▶ FILMAuthorized recall: the retrieval budget has to belong to the caller
A retrieval system must preserve authorized recall, freshness and evidence coverage while keeping token and latency costs bounded.
12 min read · Published Oct 19, 2026▶ FILMA browser is an outbound capability: how I separate destination, data and authority
Read-looking operations can send information outward through URLs, redirects, images, unfurls and fetched resources.
12 min read · Published Oct 19, 2026▶ FILMFourteen cuts through a streaming security boundary
A whole-response security decision cannot protect bytes already delivered. Define the protected unit and release boundary before choosing streaming UX.
11 min read · Published Oct 19, 2026▶ FILMWe measured local safety models. The hard part was deciding what “works” means.
Secret detection, exact redaction and content classification are different jobs. A cheap classifier that finds everything can still be unsuitable because it blocks ordinary traffic or cannot identify spans.
11 min read · Published Oct 19, 2026▶ FILMA semantic firewall needs an enforcement architecture
Semantic detection supplies a risk signal. The durable protection comes from the places that control reads, actions, disclosure and rendering even when the detector misses an attack.
11 min read · Published Oct 19, 2026▶ FILMBuild the tool broker around the authority that survives alternate paths
A model proposes a typed action; an independently authorized executor controls credentials, object access and effectful routes.
11 min read · Published Oct 19, 2026▶ FILMMake human approval survive changes in payload and context
A human approval becomes useful execution authority only when it commits to the action’s identity, payload and current effect context.
11 min read · Published Oct 19, 2026▶ FILMGive each agent turn its own authority
Keep the stable bot/service identity separate from the human authority delegated for one turn. Recompute authorization as the conversation and its audience change.
11 min read · Published Oct 19, 2026▶ FILMDesign the authority boundary between a chatbot, an assistant and an agent
Capability growth changes the system’s trust model. Public explanation, authenticated account help and autonomous action deserve distinct data paths, identities and acceptance criteria.
12 min read · Published Oct 19, 2026Name the layer before you merge the stack
MCP is tools and data. UHP draft is the product-to-harness job contract. Execution substrate is where the agent runs. Name the layer before you merge the stack.
14 min read · Published Sep 28, 2026Your hook ran. Did the forbidden action stay blocked?
How to prove an agent control protected the consequential target across tools, shells, delegation and service credentials.
15 min read · Published Sep 28, 2026▶ FILMThe demo worked. The engineering problem began afterward.
Why a convincing agent-built demo is the beginning of product ownership and engineering evidence.
14 min read · Published Sep 25, 2026▶ FILMThe agent timed out after the change went through
How to recover an agentic operation when the reply is lost after the effect begins.
15 min read · Published Sep 25, 2026▶ FILMThe agent missed the one fact that mattered
A field guide to specialist memory, source-linked context packets and intervention timing when retrieval misses a consequential condition.
12 min read · Published Sep 25, 2026▶ FILMThe schema is green. The action is wrong.
What token masks can guarantee, what typed confidence cannot prove, and where evidence, service authority and readback finish the job.
10 min read · Published Sep 25, 2026▶ FILMThe attention drift bubble
AI can generate every plan, summary and task while a decisive condition quietly disappears. Measure attention debt and preserve source-linked decisions across every handoff.
13 min read · Published Sep 25, 2026▶ FILMWhat computer-use agents can prove
A field guide to browser, desktop, and app agents: speed, blind spots, and outcome-level testing.
13 min read · Published Sep 25, 2026▶ FILMThe release gate is the product
Agent-written code can pass every test for a shortened requirement. A release needs the source product decision, exact artifact, independent evidence and a bounded customer readback.
12 min read · Published Sep 25, 2026▶ FILMHire engineers for the decisions
Architecture, attention, communication, focus, and fundamentals when agents write more of the code.
13 min read · Published Sep 25, 2026▶ FILMYour agentic workflow depends on other people’s rules
A provider refusal or changed policy can become a silent business decision unless the workflow preserves unfinished work, evidence and an authorized fallback.
12 min read · Published Sep 25, 2026▶ FILMAI in industrial control needs physical evidence
Where AI helps PLC engineering, and where simulation, safety review, and plant reality retain authority.
12 min read · Published Sep 25, 2026▶ FILMThe GPU number is not the compute product
Neocloud placement, fabric topology, sharing, queue policy, utilization, and the real cost of a completed run.
10 min read · Published Sep 25, 2026▶ FILMWhere model control actually lives
What a typed decision can actually control: full-path Jev versus small-model economics, provider-owned loops, scoped tools, context and engineering ownership.
13 min read · Published Sep 25, 2026▶ FILMModel consensus is not engineering evidence
What model debate can reveal, what polished probing cannot supply, and how real experiments turn a design into an engineering decision.
14 min read · Published Sep 25, 2026▶ FILMAgents need an independent exam
The practical controls that let a fast coding agent produce work you can actually sign off.
11 min read · Published Sep 25, 2026Podcasts
Two shows on putting AI to work.
One for engineers building AI you can trust, and one for people who use AI every day.
AI Operators by Transition 2
For engineers building AI you can trust.
AI Operators: Power User
For people who use AI every day.
02 / What I keep returning to
Questions worth
staying with.
Agents & control
Hooks, worktrees, role memory, and where instructions need to become enforceable systems.
Testing what happened
Browser and desktop agents, independent oracles, fault injection, and evidence you can sign off.
Compute in reality
Open-weight deployment, neocloud fabrics, queue policy, topology, and completed-run economics.
People & decisions
Engineering hiring, SDLC design, industrial AI, and what stays with humans as execution accelerates.
03 / Short explainers
Sixty seconds, one idea.
04 / Latest posts
What I posted this week.
A 99% approval rate tells me people clicked yes. It doesn't tell me the answer was right. Every number on a dashboard is a ruler, and each ruler measures one thing. A click measures a click. An approval measures acceptance. A completed tool…
Open on LinkedIn ↗In this queue example, a team at 50% busy has 1 job waiting. At 90% busy, 9 are waiting. Load went up less than double; the wait went up nine times. I keep people busy, never full.
Open on X ↗Our own row records intent. Only a read of the target proves the change happened. The checks I run before any record changes: - A three-way match first. - Exact decimal math, never floats. - Aggregate before joining. In one example, a raw j…
Open on LinkedIn ↗A hosted coding agent can shut down on short notice. Your code and PRs live in your code host. The agent's memories, setup scripts, schedules and session logs do not. Run its export early, and keep the file private: it holds env values. The…
Open on X ↗Release candidate Nov 15, spec freeze Dec 1, final release Dec 15. That's the published plan for the next version of the agent tool protocol spec. The release plan lists five themes. Two matter most if you run servers on that protocol in pr…
Open on LinkedIn ↗Two models agreeing on a change is a vote, not a test. Agreement tells me they read it the same way, not that it's right. The check I trust more: a seeded bad PR that the review has to block before it reviews anything real.
Open on X ↗
▶ VIDEOA provider refusal in the middle of the evidence summary. Write the case ID and draft status to durable storage. On refusal, move the same case to needs-review with original evidence and named owner. On a retry, reconcile that case instead …
Open on YouTube ↗
▶ VIDEOFelt 20% faster. Measured 19% slower. A 2025 randomized trial with 16 experienced developers is one setting, not a law, but it makes one thing clear: a survey can't be your evidence. Roll out in randomized waves and measure delivery by task…
Open on YouTube ↗
Staging drill: swap your main model provider's API key for an invalid one, then run a real task. Fails over to another model: good. Fails clearly and tells the user: fine. Hangs in a retry loop: fix it before launch. https://t.co/uqrU5FjP9…
Open on X ↗Try this in staging: swap your main model provider's API key for an invalid one, then run a real task. Three things can happen. The agent fails over to another model. It fails clearly and tells the user. Or it hangs while a retry loop keeps…
Open on LinkedIn ↗
I built two podcasts on AI you can trust, 29 episodes, now on Spotify. The host is an AI clone of my voice and the cast is fictional, AI-voiced. Every episode ends with checks you can run. The rule underneath: trust comes from measurement t…
Open on X ↗Zero leaked documents. Zero useful evidence. Both can be true. I modeled five authorized relevant documents behind 500 higher-ranked foreign documents. Global top 500 then scope filtering returned 0/5. Scope first, then ranking, returned 5/…
Open on LinkedIn ↗
Writing a working exploit chain used to be the domain of a few specialist teams. Anthropic's Frontier Red Team has now shown an unreleased internal research model doing it. Five attack chains, across Chrome, Safari, Android, Windows and Li…
Open on LinkedIn ↗
Ubicloud got Kimi K2.6's expected tool-call schema on vLLM only 92% of the time. Constrained decoding took it to 100% on Moonshot's K2VV ToolCall benchmark; a default change upstream dropped it to 91%. I'd rerun that verifier after every en…
Open on X ↗
An unreleased Anthropic model built working attack chains for Chrome, Safari, Android, Windows and Linux. Google has patched 62 Chrome CVEs from it, 13 sandbox escapes. For agents that browse, the patch release is the deadline. My check: e…
Open on X ↗
Picture an auto-reply: "I'm away until the fourth. Deliveries may be delayed." Read as prose, that's a delay notice. The headers say a machine sent it. Catch auto-replies with header rules before any model runs. A rule that is certain beats…
Open on X ↗
Cloudflare's GLM-5.3 incident returned HTTP 200 and a coherent reply holding other users' data, on prompts over 681,280 tokens. 155 responses. The check I'd run on any long-context model: a canary at max advertised context, asserting on the…
Open on X ↗
The model you call is the weights plus the engine serving them. Ubicloud wrote up serving Kimi K2.6 on vLLM. On version 0.28 they got the expected tool-call schema "only 92% of the time. In other words, 2 out of every 25 attempted tool cal…
Open on LinkedIn ↗05 / Follow the work
Every channel, one place.
04 / A little context
I’m interested in what
survives contact with reality.
I have spent more than two decades in technology, from co-founding and running a software development and marketing agency to engineering leadership roles and co-founding FrameBright, where I’m CTO. This journal is where I work through what that experience is teaching me now.
More about the road here06 / Work together
The same work,
on your system.
The journal is where I think the work through. I also do that work with other teams, in one of three shapes. If a note here sounds like a problem you have, start with the options, a call or a short note.
A project
A defined piece of work with a clear finish: an agent or AI workflow built, fixed or made ready for production.
Fractional leadership
Part-time CTO, VP of Engineering or AI leadership, for a team that needs the judgment before it needs a full-time leader.
Advisory
A regular conversation with a founder or leadership team about architecture, agents, engineering hiring and what to trust before a release.
What to send me
- What you are building, and where it stands today.
- The decision or problem you want help with.
- Which shape fits best: a project, fractional or advisory.
- Your timing, and who decides on your side.
Or write to me directly at ericmacdougall03@gmail.com.
Fixed-scope first steps, build and fix sprints and fractional CTO work run through Transition2, my AI consulting company. transition2.ai