The weights loaded. How long was the answer?

Video caption

The weights loaded. How long was the answer? The second request waits for the first to finish. Choose placement by measured experience and cost. #EricFieldNotes

Full written post / accessibility read

Here is the misleading demo: the model loads from disk, a prompt runs, and one token appears. The clip ends before the useful answer. In production, the user pays for the entire response and everyone else waiting behind it.

Now replay the same task with four users and realistic context. Record the wait before prefill, first-token delay and total completion. A home box can be the right choice for one private task and the wrong choice for a shared service; neither conclusion follows from a download screenshot.

Use a fixed task set and a timer that starts at user request. Run cold and warm, one and four users. Keep the full prompt and output length, hardware, model build, error rate and judged answer quality. Compare a smaller local model and a hosted option on identical work.

A local model earns its place when it repeatedly answers the real task within the latency and quality budget. Otherwise use a better-sized model or a capable remote service. Do this because a successful load is a setup event, not a product result.

#EricFieldNotes

Four-beat scene transcript

1. The weights loaded. How long was the answer?

Here is the misleading demo: the model loads from disk, a prompt runs, and one token appears. The clip ends before the useful answer. In production, the user pays for the entire response and everyone else waiting behind it.

Visual: A terminal banner can mask a poor user experience.

2. Queue time can exceed model time.

Now replay the same task with four users and realistic context. Record the wait before prefill, first-token delay and total completion. A home box can be the right choice for one private task and the wrong choice for a shared service; neither conclusion follows from a download screenshot.

Visual: The second request waits for the first to finish.

3. Record a complete trace.

Use a fixed task set and a timer that starts at user request. Run cold and warm, one and four users. Keep the full prompt and output length, hardware, model build, error rate and judged answer quality. Compare a smaller local model and a hosted option on identical work.

Visual: Prompt in, answer accepted, retry recovered.

4. Demand the whole answer.

A local model earns its place when it repeatedly answers the real task within the latency and quality budget. Otherwise use a better-sized model or a capable remote service. Do this because a successful load is a setup event, not a product result.

Visual: Choose placement by measured experience and cost.

Research and claim limits

• Moonshot AI: Kimi K3 official model card (S128)
• Apple: current Mac mini technical specifications (S129)

The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.