The journal
One subject at a time.
All the way down.
The longer thinking behind the videos: what I believe, what I can show, what I still need to test, and the discussion that followed.
A lucky week is not a better model
A best week, a best prompt and a good day to stop all carry luck. Compare old and new on the same cases, count only the disagreements, and set the sample size and stopping rule before you look.
Delivery you can recover
Journal before the risky step, lock the scope, refuse blind retries and leave one checked way out; then never delete agent logs you have not verified saved.
Guards that say which way they lean
How guards for commands, process stops and production deletes decide when they cannot tell, what each one leans toward, and why they are best-effort controls and not a complete boundary.
Escalation rules and the bill
Write the escalation rule as a table, count failures only from receipts the tools wrote, and put a budget wrapper with a memory around each expensive command.
The spec, the reviewers and the evidence ledger
Write down what done means before the work, bind two different reviewers to that exact text, and let a merge gate read every result on the exact commit.
Why agents cannot grade their own work
An author calling work done is a claim, not evidence; a small harness turns it into rules and checks a reviewer approval against the stage, commit and packet it actually read.
Late replies must match the current identity before they write
Four products, one bug: a late reply, cleanup step or expired lease changed current state. Carry identity into every write and test the orderings with a controlled clock.
A path contract has more readers than the review names
When a fix changes a shared path, review finds one broken reader per round. Make the reader inventory a failing test with an empty allowlist before the first fix.
A request ledger has to survive its own extractor
An audit of 236 requests found 52 stalled, and the capture fix could be silenced by one long message. How to build request capture that cannot switch itself off.
Parallel branches need one owner for the migration index
Agent branches that pick their own migration numbers collide. A landing order with one owner, reserve-first indexes and a merge-based re-chain recipe keep the chain intact.
Nothing that runs should live where cleanup can reach it
An agent harness that grew in a scratch folder lost most of its runnable code in about 80 seconds. How to spot the pattern, recover, and keep code and gates out of cleanup's reach.
When two hooks disagree, write the precedence order down
A Stop hook said push before stopping and the runner said never push. Two agents resolved it correctly, and a written precedence order keeps that from resting on judgment.
The agents used 8% of the host. The polling around them used more
Agent engines took 3.6 of 44 cores while supervisors forked 5,751 processes a second, and the idle host was disk-bound on shared storage. Measure the control plane first.
Check that each git worktree resolves to itself
One shared git setting made 17 linked worktrees write into a live tree. How to detect it with one command, prove the fix with a negative control and keep worktree isolation real.
A fail-closed allow-list can silently disable your failover
A strict model allow-list refused every request on a relay's backup path for two weeks. How to spot it, probe fallbacks with real payloads and alert on zero successes.
Fourteen control files became one release command
A site release had 14 control files and a 923 MB archive, and still could not ship. What one refusing, reading-back release script kept, and the gap it still has.
Measure the split before you redesign the agent loop
A five-second computer-use click had about 0.3 seconds of server time. Timing each verb two ways killed a push-channel redesign and found the real overhead in the helper path.
Tests that spawn processes need a group, a cap and a stop order
A loop test without its own cap left about 49,000 orphaned processes. How to contain spawning tests with process groups, an independent cap, and a cleanup order that ends the growth first.
Landed is not done: define done before agents report it
A status audit found seven false completion claims, including "proven live" beside 217 failed runs of 246. How to define done so only a live check can satisfy it.
A Rate Is a Belief With a Width: Updating From Counts to Live Traffic
Pseudo counts make a rate's uncertainty explicit, pooling steadies thin slices, and probability matching routes traffic while you learn. Worked in counts.
Counting Honestly: Ranges, Zero Failures and Paired Changes
Report every AI success rate as a range, gate automation on its lower bound, size clean test sets from the bar, and judge a change by what it flipped.
Your answer key is the product: building a gold set you can trust
A model is only as measurable as its gold set: blind, double labelled, stratified, versioned, split by time, and checked with kappa before you trust it.
Pick the metric by what a mistake does next
Precision, recall and miss rate are not a matter of taste. The harm of each wrong action picks the metric, and walking the threshold sets its value.
Counts out of a thousand: what a flag really means
A 90% detector can be wrong about most of what it flags. Count out of 1,000 at the real base rate before you let a flag block, note or route.
A 99% approval rate is not an accuracy number
A reviewer approved 99% of drafts while 1 in 12 was wrong. How to measure correctness with a blind audit, outcome truth and seeded items.
Let a business operator request a site change without handing AI a deploy token
A non-engineer can request a bounded change without becoming a proxy for unchecked code. Run generation without network or deployment credentials, enforce an exact file boundary, build and review a pinned artifact, and promote through a separate trusted deploy authority.
The offline test that spent the production quota
A fake transport is not a network boundary, and an empty negative control is not evidence. Deny egress at the test runner, keep production credentials out, prove the block with a harmless control, and preserve an unknown outcome when readback is unavailable.
The 192-core server was still waiting on one kernel worker
High CPU capacity does not guarantee progress when a serialized cleanup path stalls. Reduce sandbox churn, terminate whole process groups, add mechanism-specific early signals, and replay alarms against observed samples before retuning. Keep a local diagnosis distinct from an unverified upstream cause.
Let an agent ask for a credential without ever seeing it
Separate the agent’s request from the secret value. Bind operator input to the live run, requesting step, target and logical credential; resume the waiting work without returning the value to the transcript.
Shared Git state is the fleet coordination bug
Git worktrees isolate checked-out files but leave shared repository metadata and remote publication requiring coordination. Several observed failures had distinct causes and required different controls.
Restart is where a deletion tool can lose the only backup
A deterministic archive key and a completion receipt written after source deletion can make a retry overwrite the only backup. Model each external write, verification and destructive local effect as durable states.
When an agent writes a one-off script, it skips your safety rails
A one-off operations script can bypass approval capture and host refusal even when the underlying operation works. Make the shared CLI the policy boundary and improve it when agents reveal missing options.
What the subscription meter actually charged for tokens
API list prices were a poor proxy for subscription quota. Isolated token-class experiments measured reads, writes and output separately and changed the optimization ranking.
The five-minute cache cliff that made waiting subagents expensive
A short cache lifetime mismatched to subagents waiting behind long commands caused large prompt-prefix rewrites. Configure lifetime by worker role and verify positive and negative paths; meter savings remain unmeasured.
One per-session line broke the shared prompt cache
A per-session value in an early tool description made otherwise stable headless prompts diverge; route dynamic values only to the client that consumes them and keep remaining entropy separate.
A merge gate must prove the tests ran
A successful process exit does not prove tests executed; gates must distinguish command success, test execution, baseline comparison and unknown evidence.
A merge needs proof of no change
A similarity heuristic cannot prove reviewed work is already merged; simulate the exact merge against current main and compare the resulting tree, including reverts relative to the task fork point.
Key rotation has two readers and one writing authority
During key migration, prepare and try one decryption candidate at a time so malformed fallback configuration cannot break a valid current-key read; new writes still use one current authority.
Retire worktrees by consumer lifetime, not terminal task status
A terminal task state is only a candidate; safe worktree retirement requires the last consumer to release a shared lease, a fresh owner snapshot and verified private recovery outside the target.
Design the evidence-review screen before automating the decision
An evidence-review product must separate read-only inspection, bounded extraction proposals, conflict review and explicit human acceptance; the model does not inherit the decision right.
A read-only observer still needs backpressure
Read-only describes data mutation, not resource cost. A safe observer checks pressure during long reads, stops only its owned child, preserves last-good data and makes incomplete evidence explicit.
A fleet pause must reach the process tree
A task is not cancelled until its lifetime reaches admission waits, Git, install descendants and filesystem recovery; cancellation must leave a state the next attempt can trust.
The dependency cache belongs before the credential boundary
A dependency cache is shared mutable state. Bind identity to exact inputs, publish complete trees atomically, limit install credentials and choose deliberately between shared reuse and per-step copy.
Measure context costs before building an agent steward
Model the fleet-wide share a context change reaches before funding a Steward. Injected-text share, cached-token cost and total usage are different denominators.
A safe AI widget is a small capability system
Treat model-authored JavaScript as executable content: constrain its language, isolate browser capabilities, pin the trusted bridge and bind the serving route to the reviewed version.
Make model failover preserve the agent contract
A fallback route changes more than a model name: tool representation, role separation, budgets, privacy and refusal behavior can all differ.
Design recovery semantics into the safety state machine
A safety gate needs to preserve evidence and conservative uncertainty while distinguishing historical observations from current authority to hold execution.
Decompose work around contracts and integration costs
Use the whole-versus-pieces experiment to explain when decomposition creates useful independent boundaries and when repeated setup fragments an interdependent implementation.
Personal agents need private authority, recoverable work and measured operating budgets
Long-running personal agents need user-specific compute, storage, credentials and network authority. Multiplexed conversations alone do not create isolation.
Trace agent decisions without turning telemetry into a private-content store
Keep the links needed to reconstruct decisions and reconcile effects while minimizing stored prompts, credentials and sensitive low-entropy data.
Port agent controls across runtimes without assuming hook parity
Different CLIs, IDEs and operating systems expose different observation and denial points. Port the contract, then prove each runtime route.
Build the test harness around cross-system invariants
A test should encode the business or security promise across components, with an oracle independent of the actor and an outer containment boundary.
Separate probabilistic development from deterministic shipping
Models can propose changes and repairs. A narrow release controller can decide whether exact reviewed and tested artifacts meet a fixed contract, without reasoning its way around it.
Regional cells need one home for mutable authority
Portable containers do not make state ownership, residency and failover portable. Define one authoritative home for mutable state before adding regions.
Agent fleets need explicit ownership transfers before more workers
More workers help only when task ownership, resource admission and shared-state authority remain coherent.
Parallel AI checks need a release contract before they need more workers
Parallel checks help only when the release condition, ownership of output fields and behavior of late branches are defined in advance.
Memory success has to confirm the requester, address and receipt
Memory changes future behavior across turns. Tenant scope, namespace, origin, current permissions and deletion semantics matter as much as retrieval relevance.
A retrieval system needs to know who will read the answer
Requester authorization and answer-audience authorization are separate questions. A user may be allowed to read something that cannot be disclosed into the destination conversation.
Design agent supervision around delivery, intervention and bounded learning
Use deterministic event observation, bounded independent model judgment and a constrained intervention controller; spend review effort where it changes accepted outcomes.
Budget agent work before dispatch, then reconcile what actually happened
A token counter after completion cannot enforce a hard pre-dispatch exposure bound across parallel work and ambiguous provider outcomes.
Route AI work by the cost of accepted outcomes
Choose among routes that preserve privacy, authorization, tool semantics and required quality, then optimize expected accepted-work cost under quota and latency constraints.
Turn engineering standards into agent execution contracts
Agents need explicit constraints at admission, implementation, review and release, connected to the product decision that justified them.
Recheck current policy authority before protected bytes leave
An immutable signed policy revision is an evidence artifact. Current authorization can change while model inference or transformations are running.
Unknown is an outcome: designing agent systems around lost replies
A timeout does not determine whether an external action happened. Retrying safely requires a durable operation identity and a reconciliation path.
Durable agents need agreement between delivery, ownership and effects
A workflow is more than a queue. State changes, durable delivery, worker ownership and publication authority must agree across retries and crashes.
Where JEV earned a decision—and where we replaced it
Evaluate decision systems per class and effect: dynamically constrain candidates, use JEV where measured, and replace or escalate it when deterministic scenarios or stronger models own the uncertainty better.
Authorized recall: the retrieval budget has to belong to the caller
A retrieval system must preserve authorized recall, freshness and evidence coverage while keeping token and latency costs bounded.
A browser is an outbound capability: how I separate destination, data and authority
Read-looking operations can send information outward through URLs, redirects, images, unfurls and fetched resources.
Fourteen cuts through a streaming security boundary
A whole-response security decision cannot protect bytes already delivered. Define the protected unit and release boundary before choosing streaming UX.
We measured local safety models. The hard part was deciding what “works” means.
Secret detection, exact redaction and content classification are different jobs. A cheap classifier that finds everything can still be unsuitable because it blocks ordinary traffic or cannot identify spans.
A semantic firewall needs an enforcement architecture
Semantic detection supplies a risk signal. The durable protection comes from the places that control reads, actions, disclosure and rendering even when the detector misses an attack.
Build the tool broker around the authority that survives alternate paths
A model proposes a typed action; an independently authorized executor controls credentials, object access and effectful routes.
Make human approval survive changes in payload and context
A human approval becomes useful execution authority only when it commits to the action’s identity, payload and current effect context.
Give each agent turn its own authority
Keep the stable bot/service identity separate from the human authority delegated for one turn. Recompute authorization as the conversation and its audience change.
Design the authority boundary between a chatbot, an assistant and an agent
Capability growth changes the system’s trust model. Public explanation, authenticated account help and autonomous action deserve distinct data paths, identities and acceptance criteria.
Agents need an independent exam
The practical controls that let a fast coding agent produce work you can actually sign off.
Model consensus is not engineering evidence
What model debate can reveal, what polished probing cannot supply, and how real experiments turn a design into an engineering decision.
Where model control actually lives
What a typed decision can actually control: full-path Jev versus small-model economics, provider-owned loops, scoped tools, context and engineering ownership.
The GPU number is not the compute product
Neocloud placement, fabric topology, sharing, queue policy, utilization, and the real cost of a completed run.
AI in industrial control needs physical evidence
Where AI helps PLC engineering, and where simulation, safety review, and plant reality retain authority.
Your agentic workflow depends on other people’s rules
A provider refusal or changed policy can become a silent business decision unless the workflow preserves unfinished work, evidence and an authorized fallback.
Hire engineers for the decisions
Architecture, attention, communication, focus, and fundamentals when agents write more of the code.
The release gate is the product
Agent-written code can pass every test for a shortened requirement. A release needs the source product decision, exact artifact, independent evidence and a bounded customer readback.
What computer-use agents can prove
A field guide to browser, desktop, and app agents: speed, blind spots, and outcome-level testing.
The attention drift bubble
AI can generate every plan, summary and task while a decisive condition quietly disappears. Measure attention debt and preserve source-linked decisions across every handoff.
The schema is green. The action is wrong.
What token masks can guarantee, what typed confidence cannot prove, and where evidence, service authority and readback finish the job.
The agent missed the one fact that mattered
A field guide to specialist memory, source-linked context packets and intervention timing when retrieval misses a consequential condition.
The agent timed out after the change went through
How to recover an agentic operation when the reply is lost after the effect begins.
The demo worked. The engineering problem began afterward.
Why a convincing agent-built demo is the beginning of product ownership and engineering evidence.
Your hook ran. Did the forbidden action stay blocked?
How to prove an agent control protected the consequential target across tools, shells, delegation and service credentials.
Name the layer before you merge the stack
MCP is tools and data. UHP draft is the product-to-harness job contract. Execution substrate is where the agent runs. Name the layer before you merge the stack.
The daily record
Every angle has
room to breathe.
Platform editions are tailored to their audience. Their complete written reads are here now; players appear only for verified public videos.