Local inference needs a service-level comparison.

Video caption

Local inference needs a service-level comparison.

Queueing, rescues and failures move cost elsewhere.

Same cases, concurrency, outcome rubric and total cost.

My rule: Buy capacity only after the task class is clear.

#EricFieldNotes

Full written post / accessibility read

A company evaluating local AI should compare delivered work, not model ownership in isolation. A smaller local model, a hosted frontier API and a dedicated cluster have different privacy, latency, operating and quality profiles. Define the task envelope before comparing prices.

Suppose the local path seems cheap per device hour but needs frequent manual recovery, fails long cases and queues at shift change. The invoice still looks good while employee time and lost work rise. The reverse can also occur when a well-sized local model avoids expensive remote calls.

Pin exact model and runtime versions. Include warm and cold requests, representative long context, p ninety-five completion, output rate, successful retries, human rescue and independent quality review. Allocate labor, power, queue time and provider fees to accepted requests.

Keep a local route for work it handles safely and promptly, and send harder work where it passes the same contract. Do this because ownership, privacy and service quality are all real constraints; one benchmark number cannot collapse them.

#EricFieldNotes

Four-beat scene transcript

1. Local inference needs a service-level comparison.

A company evaluating local AI should compare delivered work, not model ownership in isolation. A smaller local model, a hosted frontier API and a dedicated cluster have different privacy, latency, operating and quality profiles. Define the task envelope before comparing prices.

Visual: The relevant alternative is not a file on disk.

2. The cheap hourly line hides the bill.

Suppose the local path seems cheap per device hour but needs frequent manual recovery, fails long cases and queues at shift change. The invoice still looks good while employee time and lost work rise. The reverse can also occur when a well-sized local model avoids expensive remote calls.

Visual: Queueing, rescues and failures move cost elsewhere.

3. Run a paired service test.

Pin exact model and runtime versions. Include warm and cold requests, representative long context, p ninety-five completion, output rate, successful retries, human rescue and independent quality review. Allocate labor, power, queue time and provider fees to accepted requests.

Visual: Same cases, concurrency, outcome rubric and total cost.

4. Place by the acceptance envelope.

Keep a local route for work it handles safely and promptly, and send harder work where it passes the same contract. Do this because ownership, privacy and service quality are all real constraints; one benchmark number cannot collapse them.

Visual: Buy capacity only after the task class is clear.

Research and claim limits

• Moonshot AI: Kimi K3 official model card (S128)
• Apple: current Mac mini technical specifications (S129)

The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.