Repository navigation
host: the last five text bounds in core/src/host still cut on UTF-16 units (steward, approvals, hooks, agents, schedules) - #2000
Conversation
f3ca07b to
29e3a3a
Compare
|
Verdict: PASS at 29e3a3a Ran at this tip: Red first, and red exactly where it should be: on the base blobs the five new What I checked:
Not run: |
UTF-16 上限清单(归零审计,
|
| 输入 | 现在 | 修后(按 fits 推) |
|---|---|---|
clip('😀'.repeat(300), 400, 'r') |
200 个 emoji + … (cut; r)(410 units / 210 码点) |
原样返回(300 字符 ≤ 400) |
clip('😀'.repeat(201), 400, 'r') |
200 个 emoji(砍 1 个) | 原样返回 |
clip('😀'.repeat(201), 200, 'r') |
100 个 emoji | 恰好 200 码点 |
excerpt('😀'.repeat(300), '', 400) |
200 个 emoji + … |
原样返回 |
也就是一段 300 个字符的文本,被一个 400 个字符的上限砍掉三分之一。这条经 read_own_chat 的 list / around / handoff 行直接给到用户和 Bot 自己,parts.js:344/:355/:465 与 chat-index.js:213/:310 都是它的消费点。
A — 判据与切点一起改(7)
| 文件:行 | 现在的判定 | 建议 | 面 |
|---|---|---|---|
core/src/host/text.js:61(clip) |
实测 单元判(text.length > max)、码点切(codePointEnd) |
判据改 fits,切点不动 |
显示 —— 清单首行 |
core/src/host/text.js:47(excerpt) |
实测 同样单元判、码点切;max 默认 OWN_CHAT_LINE_MAX = 400(host/constants.js:72) |
判据改 fits |
显示(搜索结果行) |
core/src/host/parts.js:344 |
读码 无自有判定,clip(line.text, RECENT_LINE_MAX, …) |
随 text.js:61 |
显示(recent 行) |
core/src/host/parts.js:355 |
读码 clip(line.text, LIST_LINE_MAX, …) |
同上 | 显示(listed 行) |
core/src/host/parts.js:465 |
读码 clip(line.text, RECENT_LINE_MAX, …) |
同上 | 显示(handoff keptMessages) |
core/src/host/chat-index.js:213 |
读码 rowLine 里的 excerpt(text, wanted, max) |
随 text.js:47 |
显示(搜索行) |
core/src/host/chat-index.js:310 |
读码 lines.push(... excerpt(line.text, wanted)) |
同上 | 显示(搜索命中) |
B — 只修切点、判据不动(4)
| 文件:行 | 现在的判定 | 建议 | 面 |
|---|---|---|---|
core/src/host/steward.js:49 |
实测 单元判 text.length <= STEWARD_LINE_MAX;head = ceil(1500·2/3) = 1000,头尾都用 slice,head 落进代理对就留孤立代理 |
两个切点换 cut(...).text,判据与 head 算法原样 |
模型输入(checkpoint 视图) |
core/src/host/steward.js:51 |
实测 省略数 text.length - STEWARD_LINE_MAX 是 units(同一段 2602-unit 输入显示 1102 characters left out) |
同一处计数换码点 | 模型输入(提示文案) |
core/src/host/plugin-model.js:64 |
读码 + 算术实测 String(input.system).slice(0, caps.maxSystemChars),单元切、无边界回退 |
改 cut(...).text |
模型输入(系统提示) |
core/src/client/diagnostics.js:21、:40、:92、:93 |
读码 + 算术实测 四处裸 slice:error.message 300、target/from 60 与 stack 300、what.trim() 2000、title 80 |
四处改 cut(...) 切 |
显示(诊断条目与 bug 报告预填文本) |
B 组复现(每处都把原表达式原样搬进探针):
steward:49——'a'*999 + 😀 + 'b'*1601(2602 units),head = 1000落在😀前半:头段以孤立高代理收尾,UTF-8 往返后 1 个 U+FFFD。plugin-model:64——maxSystemChars = 100,'a'*99 + 😀 + 'b'*50:slice(0,100)以孤立高代理结尾,UTF-8 往返 1 个 U+FFFD。client/diagnostics:92——WHAT_MAX = 2000,'a'*1999 + 😀 + ' tail':slice(0,2000)以孤立高代理结尾,拼进URLSearchParams后 URL 里出现%EF%BF%BD(GitHub 预填正文里就是「�」)。
C — 单独议(14)
C1 wire 契约(6) —— 改单位会改对端看到的字节边界
| 文件:行 | 现状 |
|---|---|
core/src/shared/room-share.js:58 |
wire().length > ROOM_SHARE_TEXT_MAX(裁 history) |
core/src/shared/room-share.js:88 |
wire().length(notice) |
core/src/shared/room-share.js:94 |
wire().length(notice 放不下就删) |
core/src/shared/room-share.js:104 |
body.text.length + wire().length(text 循环) |
core/src/shared/room-share.js:115 |
out.length(最终断言) |
core/src/host/room-membership.js:1041 |
text.length < 1 || text.length > ROOM_SHARE_TEXT_MAX(对端报 1–4000 chars) |
C2 模型输入(2)
| 文件:行 | 现状 |
|---|---|
core/src/host/memory.js:1270 |
本地重写的 clip:text.length > max ? cut(text,max).text + ' …(cut)' : text —— 与 text.js:61 同病同形,只差措辞 |
core/src/host/memory.js:1272-1278 |
ends:尾段走 slice(codePointStart(...))、头段走 cut,切点已码点安全,只有判据在单元上(只多截) |
C3 准入与校验(3) —— 改单位会改「收不收」
| 文件:行 | 现状 |
|---|---|
core/src/host/plugin-model.js:63 |
prompt.length > caps.maxPromptChars → prompt-too-long(只判不收) |
core/src/host/secrets.js:255 |
value.length < MIN_VALUE,文案写 "characters";改单位会让同一把旧 key 从可存变不可存 |
core/src/host/secrets.js:256 |
value.length > MAX_VALUE,同上(只提措辞) |
C4 预算大小(3) —— 只影响多留/少留几行
| 文件:行 | 现状 |
|---|---|
core/src/host/room-context.js:139-140 |
used + gap + text.length > budget 判单元;:143 截断走 cut(text, room-1),切点已安全 |
core/src/host/memory.js:266 |
used + entry.text.length > budget(记忆索引预算) |
core/src/host/memory.js:1355 |
used + text.length > REVIEW_TEXT_MAX(回顾批次预算) |
D — 保持(8)
| 文件:行 | 现状 | 为什么保持 |
|---|---|---|
core/src/host/busy-ack.js:72 |
[...clean].length > ACK_MAX_CHARS |
码点判 |
core/src/host/hooks.js:734 |
codePoints(text) > MAX_SEND_TEXT |
码点判(#1895) |
core/src/shared/remote-label.js:8 |
cut(text, 60).text |
码点切 |
core/src/host/diagnostics.js:52-53 |
cut(out, TEXT_MAX) + kept.cut |
码点切、有 cut 标志 |
core/src/host/room-context.js:284 |
lines.join('\n').length > ROOM_PRESENCE_BUDGET |
整行增删,不在字符区间切 |
core/src/client/cells.js:181(+:184) |
line.length <= PEEK_MAX 判单元,但按 grapheme 累加 |
不会劈代理对,只多截 |
core/src/host/project-assistant.js:134 |
title.length > 80 判单元,切用 cut(title, 77) |
只多截(切点已由 #2005 修好) |
core/src/client/details-drawer.js:96 |
text.length > 240 决定是否折叠 |
不切字符,只影响「展开」是否出现 |
两处容易混的
- 两个
diagnostics.js:core/src/host/diagnostics.js:52是cut(out, TEXT_MAX)+:53 kept.cut,已经码点安全 → D;core/src/client/diagnostics.js的:21/:40/:92/:93是裸slice→ B。按文件名搜会撞在一起,按路径看。 room-context.js两处不是一回事::139-140判单元、:143已用cut截(C4);:284是整行预算、不切字符(D)。
下一步
core/src/shared/text-boundary.js已有codePointStart/codePointEnd/cut(cut先toWellFormed(),返回{text, cut});缺的是fits(text, max)(= 码点数 ≤ max)。A 组用它判、仍用codePointEnd/cut切;B 组只换切点。做成一条表驱动检查(「上限 → 消费点」),新加一处按单元的判定就以「文件:行」报红——那张单开好后在本评论补链接。- C 组本轮不动:wire 改单位会改对端字节边界,准入/校验改单位会改「收不收」,预算改单位会改可容纳行数——三类都要单独判。
- 本 PR(host: the last five text bounds in core/src/host still cut on UTF-16 units (steward, approvals, hooks, agents, schedules) #1993)修的五处都在这族的切点半边;剩下的是判据半边,D 组是已经正确的写法。
Fixes #1993
Owner: Dev-2
Reviewer: Birfy
Written-by: Dev-2
Claim-Id: 9d717c402114410994aa3c6fcbec7118
What changed
Five bounds in
core/src/host/cut with a bareslice, so a bound that landed between the halves of a surrogate pair stored a lone high surrogate — read as U+FFFD by any encoder, and shown that way:core/src/host/steward.js:148text.slice(0, 80)cut(text, 80).textcore/src/host/approvals.js:75reasonString(request.reason ?? '').slice(0, 500)cut(String(request.reason ?? ''), 500).textcore/src/host/hooks.js:1041…trim().slice(0, 240)cut(…trim(), 240).textcore/src/host/agents.js:169AGENT_SESSION_TITLE_MAX(80)clean.slice(0, AGENT_SESSION_TITLE_MAX)cut(clean, AGENT_SESSION_TITLE_MAX).textcore/src/host/schedules.js:282reasonreason.slice(0, 200)cut(reason, 200).textcutiscore/src/shared/text-boundary.js's helper, re-exported bycore/src/host/text.js; each file imports it from./text.jsnext to its other host helpers (hooks.jsalready imported it). None of the five maxima changes, and the neighbours the issue names —approvals.js:80,hooks.js:928/:939/:943,schedules.js:316— are untouched.generatere-made the mirrors:patches/ds-bot/0001-core-fixes.patch(the fourcore/src/host/files that live in a topic) and the whole-file mirrorspatches/ds-bot/files/src/host/hooks.jsandpatches/ds-bot/files/test/flybot.test.js;patches/ds-bot/topics.jsongainstest/agents.test.jsunder0001-core-fixes, since that upstream file had no topic andgeneraterefuses to write it blind.Tests
Five cases, one per site, each driving the real entry point and each asserting the stored value is well formed, is the code-point prefix of the input, and — as a control — that a value under the bound is stored word for word:
core/test/steward.test.jscore/test/group-status.test.jscore/test/flybot.test.jscore/test/agents.test.jscore/test/schedules.test.jsRed first, each failing on its own. Run against the unfixed source of base
536f49f6(those five files are byte-identical on42b638c2and536f49f6), one file at a time:Each case fails in its own file and on its own site; with all five fixed the same five cases are green, and the controls stay word for word.
Verification at the tip
Tip SHA: 29e3a3a (tree
3b3465a8dfaf9ca8a74076e86260d7dfe032724f, basecf810a7d1e6993f2609e97da604db07bf96f18ff)Verified commit
9c6751e7a1ae80524fe7b7d3a02a091b2be45aadhas that same tree; the pushed head above is the tree rebuilt on the base by the fork push helper, so the two commits differ only in parentage.Ran, on a clean tree (
git status --porcelainempty) at the verified commit:verifyexit 0: core 2394 tests / 2394 pass / 0 fail (base 2389 plus these five cases);plugins/social541 (540 pass, 1 skip);plugins/tree127 (126 pass, 1 skip);plugins/timeline135 (134 pass, 1 skip);plugins/workflow336;.github/scripts563; interop 37; live-e2e 24 — 0 failures anywhere.arch-docs: OK (17 changed file(s)), satisfied by the two doc lines (docs/subsystems/core-host.mdfor the five host bounds,docs/subsystems/plugin-runtime.mdfor the mention receipt) rather than aDocs-not-needed:trailer. Lint: 0 errors (core 73 warnings, social 36, timeline 9, tree 2, workflow 16; interop 1 pre-existing warning atdev/interop/isolation/probe.mjs:130; live-e2e 0). The changelog fragment ischangelog.d/1993-p2-c5-remaining.fixed.md.Not run: the smoke test (
core/dev/bench.sh smoke …) — it needs a free bench port and a DSH checkout on this box, and this change only swaps five string bounds whose paths the five new cases drive directly.Checkpoint
{ "pr": 2000, "claim": "Dev-2", "issue": 1993, "summary": "Five bounds in core/src/host cut text with a bare slice, so one landing between the halves of a surrogate pair stored a lone high surrogate that any encoder reads as U+FFFD. They were a checkpoint tail note's quoted first line (80), an approval record's reason (500), a mention receipt's reason line (240), a DSH Agent Session's title (80) and a schedule run's failure reason (200). Each now goes through the shared cut() with its maximum unchanged. Five cases, one per site, drive the real entry point and each was red on its own before the fix.", "files": [ "core/src/host/steward.js", "core/src/host/approvals.js", "core/src/host/hooks.js", "core/src/host/agents.js", "core/src/host/schedules.js", "core/test/flybot.test.js", "docs/subsystems/core-host.md", "changelog.d/1993-p2-c5-remaining.fixed.md" ], "checks": { "ran": "TZ=UTC ARCH_DOCS_BASE=cf810a7d npm run verify; node scripts/ds-bot-patches.mjs check; canonical", "result": "green", "at": "29e3a3a5678e4b2be5cebf79adfaeadfd4d74051" }, "next": "none — ready for review" }