Hybrid inference is an allocation problem.

Video caption

Hybrid inference is an allocation problem.

Peak demand and exceptions change the path.

Decision class, model, authority, SLO and escalation.

My rule: Keep policy and outcome outside the model choice.

#EricFieldNotes

Full written post / accessibility read

A business with local and hosted AI needs a routing policy as concrete as a compute scheduler: which tasks may leave the site, which need high-end reasoning, what latency is tolerable and where uncertain answers go. An average benchmark cannot make those choices.

A compact local model may handle repetitive private work beautifully until several users arrive together or an unfamiliar case appears. A hosted model can be stronger but may violate a data restriction or depend on network availability. The router must treat both constraints as real.

For each class, pin candidate model versions and hardware, permitted data, latency and quality thresholds, fallback and owner. Replay representative cases through each full path, including connectivity loss and peak load. Record accepted task rate and human rescue cost by slice.

Send work to the cheapest eligible path that meets quality, privacy and latency, with escalation when none does. Do this because hybrid value comes from choosing well at each boundary, not simply having both local and cloud subscriptions.

#EricFieldNotes

Four-beat scene transcript

1. Hybrid inference is an allocation problem.

A business with local and hosted AI needs a routing policy as concrete as a compute scheduler: which tasks may leave the site, which need high-end reasoning, what latency is tolerable and where uncertain answers go. An average benchmark cannot make those choices.

Visual: One model should not receive every task by default.

2. The local win has a capacity ceiling.

A compact local model may handle repetitive private work beautifully until several users arrive together or an unfamiliar case appears. A hosted model can be stronger but may violate a data restriction or depend on network availability. The router must treat both constraints as real.

Visual: Peak demand and exceptions change the path.

3. Build a versioned route table.

For each class, pin candidate model versions and hardware, permitted data, latency and quality thresholds, fallback and owner. Replay representative cases through each full path, including connectivity loss and peak load. Record accepted task rate and human rescue cost by slice.

Visual: Decision class, model, authority, SLO and escalation.

4. Allocate by measured service.

Send work to the cheapest eligible path that meets quality, privacy and latency, with escalation when none does. Do this because hybrid value comes from choosing well at each boundary, not simply having both local and cloud subscriptions.

Visual: Keep policy and outcome outside the model choice.

Research and claim limits

• Moonshot AI: Kimi K3 official model card (S128)
• Apple: current Mac mini technical specifications (S129)

The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.