RAG needs an evidence contract.

RAG needs an evidence contract.

Video caption

RAG needs an evidence contract.

Brief and exception register have different authority.

It announces evolving exceptions at cutover.

My rule: Count false greens, false holds and review cost.

#EricFieldNotes

Full written post / accessibility read

I would not discard retrieval because one relevant result misses an exception. Define the cutover decision first: product authorization, affected cohorts, reconciliation, regional permission and target readback. Design discovery around those source classes.

A query for migration approval may rank the brief correctly and miss a backfill row keyed by cohort. Relevance to the user's phrase is a weak proxy for completeness at a consequential action.

The role holds the current exception map and sends a compact packet when cutover is proposed. The host resolves protected records and the service requires its own receipt. Retrieval still finds candidate documents.

Do that because a retrieval benchmark cannot tell a release owner whether the right cohort was held. Label protected cases independently and compare the full action path with a simpler prerequisite query.

#EricFieldNotes

Four-beat scene transcript

1. RAG needs an evidence contract.

I would not discard retrieval because one relevant result misses an exception. Define the cutover decision first: product authorization, affected cohorts, reconciliation, regional permission and target readback. Design discovery around those source classes.

Visual: A good search result can answer the wrong decision question.

2. Recall depends on the question.

A query for migration approval may rank the brief correctly and miss a backfill row keyed by cohort. Relevance to the user's phrase is a weak proxy for completeness at a consequential action.

Visual: Brief and exception register have different authority.

3. Give a steward narrow responsibility.

The role holds the current exception map and sends a compact packet when cutover is proposed. The host resolves protected records and the service requires its own receipt. Retrieval still finds candidate documents.

Visual: It announces evolving exceptions at cutover.

4. Grade the complete path.

Do that because a retrieval benchmark cannot tell a release owner whether the right cohort was held. Label protected cases independently and compare the full action path with a simpler prerequisite query.

Visual: Count false greens, false holds and review cost.

Research and claim limits

• Anthropic contextual retrieval technical note (S209)
• Microsoft Research GraphRAG paper (S210)
• OpenAI Agents SDK handoffs (S207)

The examples identified as illustrative or simulated are design probes, not reported incidents. Vendor specifications do not establish workload performance.