Skip to content

fix(crag): drop caller cache_control on native history blocks and stop failover on CRAG_CACHE_CONTROL (#1120) - #1129

Closed
panjinhe wants to merge 3 commits into
dofastted:mainfrom
panjinhe:fix/crag-caller-cache-anchor
Closed

panjinhe wants to merge 3 commits into
dofastted:mainfrom
panjinhe:fix/crag-caller-cache-anchor

Conversation

@panjinhe

@panjinhe panjinhe commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1120

Problem

Claude Code 2.1.325 places a cache_control breakpoint on the previous assistant tool_use as well as on the trailing tool_result. That tool_use is a model-emitted block and carries caller: {type: "direct"}. CRAG 3.0.7 blocks the body locally, before dispatch, with:

CRAG_CACHE_CONTROL: Caller cache anchor changed in native history

The official API accepts the same body.

The kernel applies the same block to a breakpoint on a tool_result whose content holds a tool_reference (the result of a ToolSearch call). In production since 10:00 UTC on 2026-11-16, all 3 requests whose trailing breakpoint landed on such a result were blocked. None of the other 1,063 CRAG requests were.

  • In production, 30 of 31 requests with cache_control on a block carrying caller were blocked. All 320 requests without one passed. Replaying a blocked body with either the cache_control or the caller removed returns 200.
  • vm2api saw the block as an empty, incomplete hop and replayed the request on up to 17 slots. Every slot runs the same check, so each replay waited ~30 s and failed the same way, and the client finally got 502 Assistant hop ended….

Fix

  • crag-print-prompt.mjs: dropNativeBlockCacheAnchors removes cache_control only from assistant blocks that carry caller. The trailing tool_result breakpoint still covers the whole prefix, so caching is unchanged. It also removes cache_control from a user tool_result that holds a tool_reference. That turn then writes no new cache entry; the next turn's trailing breakpoint covers it again. Breakpoints on other tool_result blocks and on user text are kept. The caller body is not mutated.
  • go-worker-client.mjs: a CRAG result that is policy_blocked with a body-deterministic code (currently CRAG_CACHE_CONTROL) comes back as 502 crag_policy_blocked, carrying the kernel's code and message, instead of an empty hop. callGoWorker / streamGoWorker wrap the raw calls for this.
  • upstream-error-policy.mjs: crag_policy_blocked is a request-level stop, so the pool does not replay the same body on other slots.

The anchor drop is a workaround until the kernel accepts the breakpoint. The policy-block stop is useful on its own for any future deterministic CRAG block.

Tests

  • crag-native-anchor.test.mjs: only caller blocks and tool_reference results lose cache_control, other breakpoints are kept, and the input is not mutated.
  • crag-policy-block.test.mjs: the block is surfaced as crag_policy_blocked, the pool stops without failover, and non-deterministic or committed results pass through unchanged.
  • Full unit suite passes on this branch.

Verified in production

Running since 2026-11-16 on 1.4.112 with these files mounted. Two originally blocked request bodies replay as 200. The tool_reference case went to production at 17:22 UTC. Its blocked body returned 502 before the change and 200 after it, sent unmodified.

CRAG 3.0.7 blocks a breakpoint on an assistant block that carries `caller`
(CRAG_CACHE_CONTROL 'Caller cache anchor changed in native history'). Claude
Code 2.1.325 puts one on the previous tool_use next to the trailing
tool_result; the official API accepts it. Drop only that anchor until the
kernel accepts it.
CRAG blocks the body before dispatch ('Caller cache anchor changed in native
history'). vm2api saw an empty incomplete hop and replayed it on up to 17 slots,
all of which run the same check. Surface the block as crag_policy_blocked and
classify it as a request-level stop.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T12:02:44.528044Z 73806c3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 73806c3195

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +2000 to +2002
code: 'crag_policy_blocked',
message: `Gateway kernel rejected the request before dispatch (${block.code}: ${block.message})`,
details: { retryable: false },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the policy-block code through public error mapping

When a CRAG block reaches either the non-streaming response path or the uncommitted streaming path, handle-protocol.mjs passes it through mapProtocolClientError() and mapUpstreamError(). The latter has no case for crag_policy_blocked, so its fallback replaces this code with upstream_error and discards details.retryable: false. Thus API callers receive a generic 502 without the machine-readable indication that this is a deterministic, non-retryable rejection. Add a corresponding public error mapping so the new code and retryability survive beyond the transport helper.

Useful? React with 👍 / 👎.

…RAG print

CRAG 3.0.7 rejects a breakpoint on a ToolSearch result (tool_result holding a
tool_reference) with CRAG_CACHE_CONTROL "Caller cache anchor changed in native
history", same as on a caller-marked assistant block. Replayed the blocked
request: 502 as sent, 200 with that breakpoint removed.
@dofastted

Copy link
Copy Markdown
Owner

CRAG 3.1.4 已保留 direct 工具调用及结构化工具结果上的合法缓存断点,仍执行 TTL 顺序和四断点上限;不需要删调用方 cache_control 的 workaround。确定性 policy block 的错误透传和不换槽仍归 vm2api,本版未覆盖;PR 保留,建议只评审独立调度部分。见 Release 与 #1160。

@dofastted

Copy link
Copy Markdown
Owner

感谢贡献。替代修复 #1162 已合入 main(1881d65f01)。CRAG 3.1.4 原生保留合法缓存断点;面板仅识别可信、未提交的结构化 CRAG_CACHE_CONTROL 拒绝并停止换槽,不删除合法标记或暴露原始诊断。 此 PR 不再重复合并。

@dofastted dofastted closed this Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants