FIELD NOTE / TIKTOK
A fast draft can fork the grammar state.
The short film, the complete written thought, and the evidence behind it.
The TikTok conversation link will follow its public release.
A fast draft can fork the grammar state.
Video caption
A fast draft can fork the grammar state. A rejected suffix must not govern the next token. A final parse cannot explain the path that produced it. #EricFieldNotes
Full written post / accessibility read
A grammar matcher has state. With speculative decoding, draft tokens may be proposed before the target model accepts them. XGrammar documents masks for draft positions and rollback when a proposed suffix is rejected. That extra path belongs in your integration test.
In a hypothetical engine bug, the target rejects a draft suffix but the grammar matcher stays advanced. The next mask now reflects tokens that were never accepted. The model may stall or emit a structurally odd continuation even though every individual mask call looked valid.
Use a tiny grammar and a deterministic draft that intentionally proposes a wrong suffix. Verify the target's accepted prefix, matcher rollback count, subsequent valid-token set and final output. Repeat with batching where each request has its own matcher row.
When using speculative decoding with constraints, keep an accepted-token and matcher-state test. Do this because fast draft paths add transitions where the form contract can be lost before an application parser ever sees the output.
#EricFieldNotes
Four-beat scene transcript
1. A fast draft can fork the grammar state.
A grammar matcher has state. With speculative decoding, draft tokens may be proposed before the target model accepts them. XGrammar documents masks for draft positions and rollback when a proposed suffix is rejected. That extra path belongs in your integration test.
Visual: Speculative decoding adds a less obvious control path.
2. The subtle bug is stale matcher state.
In a hypothetical engine bug, the target rejects a draft suffix but the grammar matcher stays advanced. The next mask now reflects tokens that were never accepted. The model may stall or emit a structurally odd continuation even though every individual mask call looked valid.
Visual: A rejected suffix must not govern the next token.
3. Test the rejected branch on purpose.
Use a tiny grammar and a deterministic draft that intentionally proposes a wrong suffix. Verify the target's accepted prefix, matcher rollback count, subsequent valid-token set and final output. Repeat with batching where each request has its own matcher row.
Visual: Compare accepted token trace with matcher trace.
4. Test decoder state transitions, not just JSON.
When using speculative decoding with constraints, keep an accepted-token and matcher-state test. Do this because fast draft paths add transitions where the form contract can be lost before an application parser ever sees the output.
Visual: A final parse cannot explain the path that produced it.
Research and claim limits
- XGrammar: engine integration (S173)
The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.