FIELD NOTE / TIKTOK
A fast draft can fork the grammar state.
The complete written thought and the evidence behind it. The video edition will follow its public release.
The written argument is here.
This approved TikTok edition is on the journal now. Its video player and original platform link will appear after each public release is verified.
A fast draft can fork the grammar state.
Video caption
A fast draft can fork the grammar state. A rejected suffix must not govern the next token. A final parse cannot explain the path that produced it. #EricFieldNotes
Full written post / accessibility read
A grammar matcher has state. With speculative decoding, draft tokens may be proposed before the target model accepts them. XGrammar documents masks for draft positions and rollback when a proposed suffix is rejected. That extra path belongs in your integration test.
In a hypothetical engine bug, the target rejects a draft suffix but the grammar matcher stays advanced. The next mask now reflects tokens that were never accepted. The model may stall or emit a structurally odd continuation even though every individual mask call looked valid.
Use a tiny grammar and a deterministic draft that intentionally proposes a wrong suffix. Verify the target's accepted prefix, matcher rollback count, subsequent valid-token set and final output. Repeat with batching where each request has its own matcher row.
When using speculative decoding with constraints, keep an accepted-token and matcher-state test. Do this because fast draft paths add transitions where the form contract can be lost before an application parser ever sees the output.
#EricFieldNotes
Four-beat scene transcript
1. A fast draft can fork the grammar state.
A grammar matcher has state. With speculative decoding, draft tokens may be proposed before the target model accepts them. XGrammar documents masks for draft positions and rollback when a proposed suffix is rejected. That extra path belongs in your integration test.
Visual: Speculative decoding adds a less obvious control path.
2. The subtle bug is stale matcher state.
In a hypothetical engine bug, the target rejects a draft suffix but the grammar matcher stays advanced. The next mask now reflects tokens that were never accepted. The model may stall or emit a structurally odd continuation even though every individual mask call looked valid.
Visual: A rejected suffix must not govern the next token.
3. Test the rejected branch on purpose.
Use a tiny grammar and a deterministic draft that intentionally proposes a wrong suffix. Verify the target's accepted prefix, matcher rollback count, subsequent valid-token set and final output. Repeat with batching where each request has its own matcher row.
Visual: Compare accepted token trace with matcher trace.
4. Test decoder state transitions, not just JSON.
When using speculative decoding with constraints, keep an accepted-token and matcher-state test. Do this because fast draft paths add transitions where the form contract can be lost before an application parser ever sees the output.
Visual: A final parse cannot explain the path that produced it.
Research and claim limits
- XGrammar: engine integration (S173)
The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.