Skip to content

fix(crag): cap kernel session_slots at the plan-sized worker slots - #1123

Closed
panjinhe wants to merge 2 commits into
dofastted:mainfrom
panjinhe:fix/crag-session-slots-cap
Closed

panjinhe wants to merge 2 commits into
dofastted:mainfrom
panjinhe:fix/crag-session-slots-cap

Conversation

@panjinhe

Copy link
Copy Markdown
Contributor

Problem

On a CRAG v2 slot, writeKernelConfig sets kernel.json session_slots from the panel / routing default (inference.session_slots, 20). The plan-based slot capacity (#1081–#1083) sizes the kernel's slots_per_worker lower: Pro 5 (2g), Max 5x 12 (4g). crag-v2 checks the two at startup and refuses to start:

crag-v2: session_slots exceeds physical capacity

So every Pro / Max 5x CRAG slot fails its next start or restart (kernel install, restart, capacity reload, wrap → crag switch) and is left stopped with kernel_start_failed.

In production (v1.4.112), switching a Pro slot from wrap to crag hit this. The 9 CRAG slots already running survived only because they had started under an older config. Their kernel.json had been rewritten with session_slots: 20, so each would have failed on its next restart.

Change

In the crag-v2 branch of writeKernelConfig, cap session_slots at slots_per_worker:

config.session_slots = Math.min(
  resolveSessionSlots(followGlobal ? {} : target, kernelRouting),
  config.slots_per_worker,
)
  • A lower per-VM pin (for example 3 on a 5-slot kernel) is kept.
  • With slot capacity disabled, slots_per_worker is the kernel maximum (20), so behaviour is unchanged.
  • When slots_per_worker already matches, sessionPolicyChanged stays false, so no extra idle reload is queued.

Tests

  • New CRAG session_slots never exceeds the plan-sized kernel slots in test/unit/kernel-router.test.mjs. It fails without the change (actual: 20, expected: 5).
  • kernel-router, slot-capacity*, seat-planner, routing-runtime*, slot-engine and crag-runtime* unit suites: 181/181 pass on current main.
  • biome format: clean.

Production

Deployed on v1.4.112 as a mounted patch. All 12 CRAG slots keep session_slots equal to slots_per_worker (Pro 5, Max 5x 12), and a converted Pro slot starts normally.

crag-v2 refuses to start with 'session_slots exceeds physical capacity'
when kernel.json carries the panel default (20) on a Pro (5) or Max 5x
(12) slot, so every CRAG slot failed its next restart after 1.4.112.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T10:58:42.954629Z 78209e4 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 78209e44b0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1119 to +1122
config.session_slots = Math.min(
resolveSessionSlots(followGlobal ? {} : target, kernelRouting),
config.slots_per_worker,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Recompute session limits when physical slot capacity grows

If the capacity reconciler raises a VM after this config is written—for example, a Pro VM from 1g/256 pids to 2g/448 pids—the new clamp persists session_slots: 2. The watchdog/startup convergence path uses syncKernelSlotCount(), which only updates slots_per_worker, so the subsequent restart reads five physical slots but still enforces the old two-session limit. Meanwhile, the pool's seatCapOf() uses the increased effectiveSlotCount() and budgets five seats. This leaves the newly available capacity unusable until a full configuration rewrite occurs. Recompute the capped session policy during slot-count convergence as well, while preserving explicit lower overrides.

Useful? React with 👍 / 👎.

The watchdog / ensureRustKernel slot-count convergence only rewrote
slots_per_worker. A capacity raise (Pro 1g -> 2g) left session_slots at the
old cap, and a shrink left it above the new count so crag-v2 refused to
start. syncKernelSlotCount now re-caps crag-v2 session_slots at the new
count from the uncapped request (VM pin, else panel default), keeping lower
pins; without a request it only lowers.
@dofastted

Copy link
Copy Markdown
Owner

感谢贡献。替代修复 #1162 已合入 main(1881d65f01)。写入与校正旧配置同时约束 session_slots,已有超配不会因为持平槽数而跳过,保留忙槽延迟重载和 #1158 的授权准入。 此 PR 不再重复合并。

@dofastted dofastted closed this Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants