FIELD NOTE / TIKTOK
A typed model cannot choose the missing answer.
The short film, the complete written thought, and the evidence behind it.
The TikTok conversation link will follow its public release.
A typed model cannot choose the missing answer.
Video caption
A typed model cannot choose the missing answer. A clean Choice is not a customer outcome. A typed answer cannot create missing observations. #EricFieldNotes
Full written post / accessibility read
Imagine an incident router with three choices: close, page or monitor. Aggregate traffic looks healthy, but the affected tenant has sent no new requests. None of those choices means 'insufficient relevant exposure.' A perfectly typed answer still has to choose from the menu we gave it.
TypeSafe documents a Choice distribution over the supplied options. A high confidence value cannot certify recovery for a tenant absent from the sample. If the route to gather evidence is missing, tuning a threshold only makes the wrong workflow look more numerical.
Give the classifier an explicit insufficient-evidence option. Keep the affected cohort and closure criterion in an owner-controlled incident record. At dispatch, recheck that record and measured cohort exposure. If the evidence is absent or stale, the service rejects auto-closure and routes to the on-call owner.
Put a healthy aggregate dashboard beside a protected tenant with zero requests in the holdout. Require the workflow to pause, collect relevant evidence and name an owner. Do this because the form of a model answer can be guaranteed while the fact needed to act remains unknown.
#EricFieldNotes
Four-beat scene transcript
1. A typed model cannot choose the missing answer.
Imagine an incident router with three choices: close, page or monitor. Aggregate traffic looks healthy, but the affected tenant has sent no new requests. None of those choices means 'insufficient relevant exposure.' A perfectly typed answer still has to choose from the menu we gave it.
Visual: The incident may need evidence, not another verdict.
2. Confidence is conditional on the menu.
TypeSafe documents a Choice distribution over the supplied options. A high confidence value cannot certify recovery for a tenant absent from the sample. If the route to gather evidence is missing, tuning a threshold only makes the wrong workflow look more numerical.
Visual: A clean Choice is not a customer outcome.
3. Protect the missing-evidence branch.
Give the classifier an explicit insufficient-evidence option. Keep the affected cohort and closure criterion in an owner-controlled incident record. At dispatch, recheck that record and measured cohort exposure. If the evidence is absent or stale, the service rejects auto-closure and routes to the on-call owner.
Visual: The host owns current incident state and closure rights.
4. Test the zero-exposure case.
Put a healthy aggregate dashboard beside a protected tenant with zero requests in the holdout. Require the workflow to pause, collect relevant evidence and name an owner. Do this because the form of a model answer can be guaranteed while the fact needed to act remains unknown.
Visual: A typed answer cannot create missing observations.
Research and claim limits
- TypeSafe AI: Jev documentation (S98)
- TypeSafe AI: Jev 1.13 jaggedness (S99)
- TypeSafe AI: Confidence (S100)
The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.