Skip to content

feat(knowledge): unify search across cases, skills, and records - #466

Open
jiaran-king wants to merge 10 commits into
InferMatrixLab:mainfrom
jiaran-king:codex/knowledge-find-pr
Open

jiaran-king wants to merge 10 commits into
InferMatrixLab:mainfrom
jiaran-king:codex/knowledge-find-pr

Conversation

@jiaran-king

@jiaran-king jiaran-king commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

关联需求与交付范围

Refs #379

Agent 描述优化中遇到的问题后,可以在一次查询里找到适用的指南、资料记录和历史案例,再用结果给出的命令读到带出处、适用条件、限制和版本的原文。本 PR 交付这个本地检索与读回流程。

原有搜索与观察到的问题

原有入口 原有行为与影响
knowledge search / knowledge research 按已安装的记录目录搜索或展开资料。自然语言查询里的额外词会影响要求全部词匹配的旧入口。
collect --relevance 对打包 Skill 排序。查找 checkout Case 需要另外选择来源和类型。
collect / read 按来源、ID 或版本收集和读取。大 Bundle 的资源清单会挤占交给 Agent 的工具返回。

在已有回放里,相关 Case 曾在选择阶段落选。另一些已选材料在返回外层工具时出现过截断。对 Agent 来说,命中和拿到正文是两步。

本 PR 的改法

  • inferrsi find "<问题描述>" 用同一套词项排序比较已登记的 Case、Skill 和 package Record。字段权重优先考虑标题、摘要、标签和 findings,预先计算词频并降低长正文的权重。查询支持词边界、常见分隔符、中英文别名和 --kinds。
  • 从 7 份原始 Case 正文重新提炼 35 条结构化 findings,让检索可以直接命中尝试过的方向、结果、原因和复试条件。每条预览带记录中的适用范围或边界。
  • 结果提供源限定 ID 和可执行的 open,JSON 提供 open_argv。工具用同一 reader 检查返回项能否读到材料。详细的正文、来源和 revision 由 read --material --omit-resources 返回,需要清单时可以完整读取。
  • README、操作文档和插件指引以 find → read 介绍常规发现流程。旧的精确收集、读取和资源接口继续服务现有调用方。Codex 和 WorkSwarm 插件沿各自的安装入口读回。

例如:

inferrsi find "mask caching ran out of device memory" --kinds case --limit 1

结果会提示 Qwen Ascend 案例及其读取命令。读回正文时可取得 revision,供后续固定版本引用。

实测结果

下表在合并 main 后的 8c0a7bfc 上复测。类型标注只统计预期有材料的查询。

查询集及判定方式 重写前记录 8c0a7bfc
#380,期望类型出现在前三 16/17 15/17,其中 Case 11/11
基于已有 evidence 的类型标注,期望类型出现在前三 21/34 24/34,其中 Case 7/10
九条冻结开发查询,目标 Case ID 出现在混合搜索前三 — 7/9
同一组查询限定 Case,目标排第一并读回完整正文 — 9/9

冻结查询包括 Qwen(5 条)和共享资源哈希(4 条)。返回正文和冻结原文逐字节一致,来源、适用条件和限制均保留。按返回的 revision 重读,结果一致。

用同一版本对照 package:inferrsi.common.skills.optimization_search_policy 的完整和精简读回,JSON 从 223,782 字节降到 23,757 字节。正文、来源、适用条件和 revision 一致。

Qwen-Image 单次真实 Agent 检索对照

基于 #338 的缩比 qwen21-denoise-npu 题包,使用两个全新 GPT-6 Luna / max 会话,在同一张 Ascend910B2C 上串行比较旧 session knowledge 与本 PR 的 find → read;两组都开启知识库,各完成 8 轮和独立 final。A 的上下文预算为 16,384 字节,B 从预览中选择并读回正文。每次评测均覆盖五个正确性 case、其中三个计时,pairs=2, samples=3, warmups=1。

指标 A:旧检索 B:find → read
完成轮次;final 质量检查 8/8;PASS 8/8;PASS
选中候选 第 8 轮 第 6 轮
查询 / 返回项 / 正文读取 / 采用材料 / 回退 1 / 5 / 1 / 0 / 0 1 / 5 / 2 / 2 / 0
Campaign 耗时 928.1 秒 881.5 秒
输入 token,其中缓存输入 3,784,667;3,613,184 4,073,202;3,939,072
输出 token 47,606 46,008

本次新检索会话实际读取并采用了更多针对性材料:第 1 轮引用 vllm_omni.ascend.diffusion.qwen_image_21_eager_t2i(revision sha256:b1740d8dd28f3c2f7ec2e4fbb9233a7ce311f52210fa3c6365bb320ad191e142),优先测试请求内 conditioning 复用;第 2–4 轮引用 case.qwen-ascend-rejected-directions(revision sha256:039a074c881a924327549416d143fcf7cb550493e87352113b45a08d8717827a),拆开验证 RoPE 和 mask 缓存。A 送达的 package:inferrsi.ascend.skills.ascend_optimization_casebook(revision sha256:7208ca2225432e313e1f899bde5b963f39d617b45996cd336da67238f8a5fa86)没有被记录为改变候选或验证决定。采用数按唯一材料计算;A 的另外四项只交付了命中元数据。

**范围:**每组仅一次自主查询,提问、语料和交付方式不同,也没有独立相关性标注;这张表不能单独证明新检索整体更好或收益稳定。Campaign 耗时包含优化和评测,不是单次检索耗时;缓存输入包含在输入 token 中。表中 PASS 仅指题包 final 质量检查,不代表整个检索对照或真实图像生成、服务性能验收通过。正式 A 前另有一次中止的 Sol/high 会话,其三轮结果未纳入比较;两组在不同时间运行。原始证据保存在私有环境,尚未公开完整 evidence tree。

检查与结果边界

  • 用无 PyYAML 的 Python 3.11 基础 wheel 复测 cuda graph warmup。 find 命中 kernel-writing-guide,collect --relevance 重新搜索打包语料,命中 inferrsi.ascend.skills.ascend_aclgraph_capture_replay。指引要求先判断新结果是否适用,再用它的 ref、skill_documents.original 和 revision 经 resource 读回正文。两条评审查询的读回均与 wheel 原文逐字节一致,基础安装里 Case 的 find → read 也通过。 David 的评审
  • 知识检索、读取、索引、安装、模块边界和插件相关套件 286 项通过,4 项跳过。另有 2 项 DSH 检查因 Node 收到 SIGABRT 失败,同两项在 main 的 891aadda 上出现同样的失败。Ruff、diff 检查和生成索引校验通过。
  • 采用默认版本推导成功构建 wheel 和 sdist。Python 3.10 和 3.11 的干净 wheel 安装均通过检查,929 份资源与源码逐字节一致。
  • 类型命中表示前三里出现了期望类型。已登记材料的覆盖和词汇差异仍会影响排名。
  • 范围按 tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 的提议保留 find → read 作为默认查询和读取流程,旧命令用于回退。默认入口切换的决定待在 tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 记录。单次 feat: share RSI optimization management across project and benchmark tasks #338 对照观察到材料采用,稳定的检索改善和优化收益仍待验证。 #379 范围提议

知识作者声明

{
  "schema_version": 1,
  "head_sha": "8c0a7bfc39a9f95d610f6bd0b9f3682200ad0606",
  "human_only": false,
  "material_ai_authors": [
    {
      "tool": "Claude Code (earlier search implementation)",
      "model_id": "unknown",
      "effort": "unknown",
      "role": "author"
    },
    {
      "tool": "Claude Code (current-main knowledge migration)",
      "model_id": "claude-opus-5-5",
      "effort": "max",
      "role": "author"
    },
    {
      "tool": "Codex (case findings rewrite, read handoff, regression fixes, and CLI integration)",
      "model_id": "gpt-6.1-sol",
      "effort": "max",
      "role": "author"
    }
  ]
}

…PyYAML

Each casebook entry already recorded the directions tried, their outcome and
when a retry would be justified, but only as prose or a Markdown table that
ranking could not weigh. Add a `findings` list to all seven cases: direction,
outcome, recorded reason, revisit condition, retrieval keywords and `scope`.
Every text field is taken from the existing case body; no claim, cause or
measurement is added and no existing field changes.

`scope` states what would have to change for an observation to stop holding:
`local` (that host or harness), `regime_bound` (the cohort the case names) or
`general` (a property of the method). It is not derivable from the outcome --
a rejection can be bound to one resolution or hold everywhere. Of 35 findings,
24 are regime_bound, 6 general and 5 local. The source-contract cases are
regime_bound: each reads one pinned upstream commit and says so in its own
applicability, and a later revision can change the dispatch it describes.

Reading a case parses no skill frontmatter, but importing the skill catalog
required PyYAML, so a base install could not discover or open one. The parser
is now imported where frontmatter is parsed. Its absence raises
MissingSkillsExtra, a ValueError subclass in a module of its own so catching it
never imports the parser, and `collect` lets it through rather than reporting
"nothing found" for an install problem. The snapshot tool relied on the old
import-time failure to stop before side effects, and now asks for the parser
explicitly at that point.

The reader exposes `cases()` and `case_resource_path()` so a caller ranking
cases resolves them by the same rule `read` uses.

Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=max] (author)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
…ndings

Mechanical follow-up. The generator pins each card's source hashes to a
revision, so the three skills that own the edited cases -- the optimization
casebook, the kernel-writing guide and the search policy -- held hashes for
the case files as they were before findings were added, and `indexed_skill`
rejected those projections as stale. Regenerated with
tools/build_skill_index.py against the previous commit.

Agent-assisted-by: Claude Code / claude-opus-5-5 [effort=max] (author)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Port the verified find and read flow to the current Knowledge package and Apps-facing CLI. Keep query fixtures, compact exact reads, and standalone plugin coverage together.

Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Agent-assisted-by: Claude Code (author)
Agent-assisted-by: Codex (author)
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Skill evaluation

PLANNED: not executed; there are no task outcomes yet.
No approval, model efficacy or current-head test result is implied.

  • Next: run the pinned source/index --check below at head 8c0a7bfc39a9f95d610f6bd0b9f3682200ad0606.
  • Task selection: confirm the prepared task tests the changed claim.
  • Selected comparisons: 1 · Estimated task calls: 2. Uncovered/unsupported changes: 32; Canonical library paths: 6; Generated projection paths: 4; Other paths: 22. These are paths, not independent claims.
  • Return: command, input/source pin, exit status, outputs/failures and untested claims for independent verified/refuted/unverified disposition.
Author verification steps (nonpaid; not yet run)
  1. Bind the checkout. Run git rev-parse HEAD; it must equal 8c0a7bfc39a9f95d610f6bd0b9f3682200ad0606. If it differs, obtain a current plan; do not reuse old results.
  2. Check source/index consistency. At that PR checkout, using the reviewed repository tool and its documented development dependencies, run:
python tools/build_skill_index.py --source-revision 8288a004c333b0e338b3ad31fe823052a142b90a --check

The declared source pin matches this head's canonical library bytes. The command has not run: retain its exit code and full drift/success output. It checks the generated projection, not claim truth or skill efficacy.

  • optimization-search-policy
    Selected by the trusted path mapping: src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json.
    Task case-05: Diagnose this blocked request at the recorded decision boundary; propose the next read-only check.
    Scope: Synthetic finite-state observations; not a production trace or engine benchmark.
    Required answer fields: diagnosis, evidence, next_check, reuse_authorized.
    Independent check: frozen kv-admission-diagnostic-v1 grader/schema bound in details. Confirm it tests the changed claim, not merely the touched path.
    From the downloaded skill-eval-plan artifact directory, inspect the first prepared task and its required answer schema (read-only; not grading):
python -m json.tool optimization-search-policy/plan/tasks.json
python -m json.tool optimization-search-policy/plan/schema.json

Uncovered scope by location: Canonical library paths: 6; Generated projection paths: 4; Other paths: 22. Path counts are not claim counts; no metadata-only conclusion is inferred.
3. Return evidence for independent review: head, check/task/source pin, command, environment, exit status, outputs/failures and untested claims. Label scripted controls. Request each claim's disposition: verified, refuted or unverified; this plan proves neither model provenance nor benefit.
Check recipes and evidence limits

Evidence and logs · Retention: 30 days.
Retained artifacts are verifiable only before Unix time 1797432386 while the linked artifact is available. Expired evidence is unverified.

Plan bindings, budget, frozen exam and advisory details

Advisory: obtain missing evidence

Retained evidence availability: planned / unexecuted. No approval or merge authorization is issued.
Applicable only while these revisions and inputs match the current PR.
Paid execution requires trusted label authorization and cumulative PR reservation. Unknown accounting blocks new paid calls; planning is not execution.
Pipeline evidence only; offline fixed outputs do not establish skill efficacy.
Unchanged-correct is valid neutral evidence, not incremental benefit.
Content/source review and authorship admission remain separate.
Trusted regrading checks retained answers, not model/hardware provenance or a hostile executor; this is not an efficacy certification.

  • Retained evidence is planned / unexecuted.
  • Uncovered/unsupported scope needs a case or explicit reviewer disposition.
    Plan SHA-256: d1e189855f4296a8d96cb83e9abd2a7ba0a9f0e3e7584ed57439b0d298f174ff
    Base: 891aadda6d4abc050def2baa2814b284f87808e5 · Head: 8c0a7bfc39a9f95d610f6bd0b9f3682200ad0606
    Trusted implementation: 891aadda6d4abc050def2baa2814b284f87808e5

Model/settings: {"effort":"not_applicable","name":"claude-opus-5-5"}
Budget: {"comparisons":1,"estimated_calls":2,"limits_per_comparison":{"campaign_seconds":600,"max_task_calls":2,"repeats":1,"timeout_seconds":180,"token_stop_new_trials":200000},"provider_config_sha256":"b28d80f8194fbe0f5a9f01515efb24b218979fd45f702776560da276f4b30d04","reservation":{"amount":"20.34","currency":"USD","group":"default","limit":"200","per_request":"10.17","pricing_sha256":"2a82453ec18b75b92e12f9a9c18e6b57811a3ef593c31e4639b839df218cac39","requests":2}}

Skill Comparison Cases Estimated calls
optimization-search-policy update case-05 (holdout) 2

Frozen exam hashes: {"cases":"a0dd4e1615402a3a629f925c15435470095c27eeac54c4a2080c2d874abeb031","grader":"e25cfee943052aaefbddf3f545fe6d56d95ab1edb9723cd9353feeabc954ea9a","schema":"46ed2e63dadd5f9fa60107ff58b020554b42f47472ca80b4de9add79d92f8449"}

Selection/task metadata (not a relevance verdict):

{"adapter":"kv-admission-diagnostic-v1","basis":"trusted_path_mapping","matched_paths":["src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/cases/sarathi-chunk-budget.json"],"required_answer_fields":["diagnosis","evidence","next_check","reuse_authorized"],"tasks":[{"id":"case-05","provenance":"Synthetic finite-state observations; not a production trace or engine benchmark.","request":"Diagnose this blocked request at the recorded decision boundary; propose the next read-only check.","split":"holdout"}]}

Automatic execution / cumulative PR budget:

{"reason":"Trusted configuration or pricing missing, invalid or expired","remaining_usd":"unknown","reserved_or_unresolved_usd":"unknown","spent_or_legacy_usd":"unknown","state":"blocked"}

Projection check selection (not an executed result):

{"declared_in":"src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/_provenance.json","source_revision":"8288a004c333b0e338b3ad31fe823052a142b90a","status":"ready"}

Provider/settings: {"client":{"api_version":"2023-06-01","endpoint":"https://yibuapi.com/v1/messages","kind":"anthropic-messages-full-context-v1","max_output_tokens":4096,"max_read_bytes":32768,"max_request_bytes":262144,"max_requests":1,"max_tool_calls":12,"provider":"yibuapi"},"limits":{"token_stop_new_trials":200000},"model":{"effort":"not_applicable","name":"claude-opus-5-5"},"pricing":{"currency":"USD","expires_at":1793271228,"group":"default","max_input_tokens":1000000,"model":"claude-opus-5-5","rate_evidence":"https://github.com/InferMatrixLab/InferRSI/issues/215#issuecomment-6467531341","rates":{"cache_read":"0.4","cache_write":"10","input":"8","output":"40"},"request_fee":"0","unit":"per_million_tokens"},"spend_limit":{"amount":"200","currency":"USD"}}

Uncovered / unsupported paths (32)
  • uncovered: "README.md"
  • uncovered: "docs/knowledge-operations.md"
  • uncovered: "docs/operations.md"
  • uncovered: "docs/quickstart.md"
  • uncovered: "plugins/inferrsi/skills/inferrsi/SKILL.md"
  • uncovered: "plugins/inferrsi/skills/inferrsi/scripts/inferrsi.py"
  • uncovered: "plugins/workswarm/inferrsi/skills/inferrsi/scripts/inferrsi.py"
  • uncovered: "src/inferrsi/apps/cli/init.py"
  • uncovered: "src/inferrsi/apps/cli/arguments.py"
  • uncovered: "src/inferrsi/knowledge/_knowledge_errors.py"
  • uncovered: "src/inferrsi/knowledge/data/query_aliases.json"
  • uncovered: "src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/_provenance.json"
  • uncovered: "src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/inference-optimization-casebook.json"
  • uncovered: "src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/kernel-writing-guide.json"
  • uncovered: "src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/common/skills/optimization-search-policy.json"
  • uncovered: "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/admission-drain-occupancy.json"
  • uncovered: "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/drama3-rejected-directions.json"
  • uncovered: "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-ascend-rejected-directions.json"
  • uncovered: "src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/cases/qwen-b300-rejected-directions.json"
  • uncovered: "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/sglang-gdn-source-contract.json"
  • uncovered: "src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/cases/vllm-gdn-source-contract.json"
  • uncovered: "src/inferrsi/knowledge/knowledge_find.py"
  • uncovered: "src/inferrsi/knowledge/knowledge_operations.py"
  • uncovered: "src/inferrsi/knowledge/skill_catalog.py"
  • uncovered: "tests/check_installed.py"
  • uncovered: "tests/fixtures/find/README.md"
  • uncovered: "tests/fixtures/find/labelled-queries.json"
  • uncovered: "tests/test_codex_plugin.py"
  • uncovered: "tests/test_knowledge_cli.py"
  • uncovered: "tests/test_knowledge_find.py"
  • uncovered: "tests/test_knowledge_install.py"
  • uncovered: "tools/skill_eval_snapshot.py"

@jiaran-king
jiaran-king marked this pull request as ready for review November 16, 2026 08:42
Agent-assisted-by: Codex (author)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
@Dong1017

Copy link
Copy Markdown
Collaborator

Posted by an agent on behalf of @Dong1017 (incremental review only; first review on this PR, no earlier human findings to repeat).

PR review verdict — DO NOT PASS

Reviewed base 016c7c7df023944f0f46d8e777d6d31f1df1764c (current main, also the merge base) and head 80ec573eb5b635a938cc6e215f1fd2dbed8c0d7f, 32 files, +2919/−62.

The find implementation itself looks careful: case discovery goes through the same KnowledgeReader.cases() that read uses, case resource checks are shared through case_resource_path, and every returned hit is opened through the reader before an open command is offered. The two blockers below are about admission and scope, not the ranking code.

Blockers

  1. Required knowledge-model-policy check fails at this head. The PR changes protected paths (src/inferrsi/knowledge/data/skill_library/** cases, knowledge_operations.py, skill_catalog.py, docs/knowledge-operations.md; see docs/knowledge-model-policy.md "exact protected scope"). The declarations run for 80ec573e reports "kind": "noncompliant" with "Material AI author 1: model is unapproved or suspended" and "Material AI author 3: model is unapproved or suspended" (run 43834106747). Authors 1 and 3 in the PR-body block declare model_id: "unknown" / effort: "unknown", and the policy says unknown values "stay truthful and do not qualify". Expected: either an exception approval from the approval owner per docs/knowledge-model-policy.md (it allows that "explicitly unknown settings may be considered"), or a revision whose protected-path content is authored only by qualifying declarations. Please do not change the declared values unless they are actually known.

  2. The change to the default agent entry point is not covered by the governing Issue's accepted scope. The PR says Refs #379. tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379's product decision currently reads: do not promote the task: 改进真实优化任务中的材料发现与使用 #380-tested guidance to the product entry and temporarily keep the existing catalogue and rg; it also says the tracker grants experiment: compare Skill Indexer and rg on MoE with GPT-6.1 Sol high #370/task: 支持每个优化迭代的知识库命中归因统计 #344 no extra implementation authority, and experiment: compare Skill Indexer and rg on MoE with GPT-6.1 Sol high #370 is the lane that compares retrieval entry points. This PR replaces the plugin's discovery guidance (plugins/inferrsi/skills/inferrsi/SKILL.md:124-150: knowledge research / knowledge search / collect --relevance are removed in favour of find → read) and makes find the README/quickstart default (README.md:227-228, docs/quickstart.md:55-56). The PR itself states that whether agents adopt the results and any optimisation benefit are not yet verified. Expected: a requester/maintainer note on tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 accepting find as the default entry, or a narrower revision that ships find / read --omit-resources as additional commands while leaving the plugin's default discovery guidance unchanged until that decision is made.

Optional (not blocking)

  • The quickstart example does not lead with the finding it is about. docs/quickstart.md:55 uses inferrsi find 'mask caching ran out of device memory'. Locally at this head (default kinds, and also with --kinds case --limit 1 as in the PR body), the Qwen Ascend case ranks first, but the finding that answers the query ("Cache per-step masks and prompt tensors → NPU OOM", rejected) is listed third, after two no_gain findings. caching/masks do not match cache/mask and nothing maps "out of … memory" to OOM, so within the case only mask (3 of 6 findings) and device (1 finding) score, and relevant_findings (src/inferrsi/knowledge/knowledge_find.py:218-238) puts the fusion-attention finding first. test_findings_are_ordered_by_query_overlap_and_truncated (tests/test_knowledge_find.py:136-143) asserts the right order only for the different wording cache per-step masks ran out of device memory. Either use that wording in the quickstart, or add an out of memory → oom synonym in query_aliases.json and pin the documented example in a test.
  • full-check (3.10) was still pending when this was written; full-check (3.11), installed-package (3.10/3.11) and pr-references were green.

Not verified: the ranking-quality table in the PR body (16/17, 21/34, 7/9, 9/9) and the 210,721 → 10,950 byte read-back reduction; I did not rerun the labelled-query fixtures against #380.

Checking scope: read the PR body, #379, the CI check list and the failed declarations log, docs/knowledge-model-policy.md, and the full diff of knowledge_find.py, cli.py, arguments.py, knowledge_operations.py, skill_catalog.py, _knowledge_errors.py, the plugin SKILL.md and README.md against 016c7c7d; skimmed the case JSON additions and query_aliases.json. Ran in a clean venv at the head (pip install -e '.[skills]'): pytest tests/test_knowledge_find.py tests/test_knowledge_cli.py tests/test_codex_plugin.py → 118 passed; ran the quickstart query and the printed open command (returns material and revision with resources omitted), and find --limit 0 (rejected with a clear error). Did not run the full suite, Ruff, the Windows/PowerShell path, the WorkSwarm/DSH plugin checks, or any agent-adoption evaluation.

Copy link
Copy Markdown
Collaborator

审阅了本 PR 的 base 016c7c7df023944f0f46d8e777d6d31f1df1764c 到当前 head 80ec573eb5b635a938cc6e215f1fd2dbed8c0d7f(完整 5 个提交)。当前 head 上不能 Approve,我和 @Dong1017 在评论 6911930950 中的两点阻塞结论一致,核对如下。

  1. 必选检查 knowledge-model-policy 在该 head 上为 noncompliant。declarations 日志写明:Material AI author 1: model is unapproved or suspended,Material AI author 3: model is unapproved or suspended(run 43834106747)。PR 正文声明里这两位的 model_id / effort 均为 unknown。按 docs/knowledge-model-policy.md,未知值应如实填写,但不能通过准入;需要已批准模型且 effort 为 xhigh 或 max,或由批准人按规范给出 exception。请不要为了过检而改写并不确实知道的声明值。该必选检查通过之前,这个 head 无法合入。

  2. 治理 issue tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 仍是 status:proposed,且当前产品决定是暂不把 task: 改进真实优化任务中的材料发现与使用 #380 所测短指引推广为产品入口,并临时保留现有目录和 rg;跟踪项也写明并不授予 experiment: compare Skill Indexer and rg on MoE with GPT-6.1 Sol high #370/task: 支持每个优化迭代的知识库命中归因统计 #344 额外实施权限。本 PR 却把插件默认发现路径改成了 find → read(plugins/inferrsi/skills/inferrsi/SKILL.md),并在 README / docs/quickstart.md 中把 find 写成默认入口。请先在 tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 上由请求者/维护者明确接受 find 作为默认入口,或把这次交付收窄为新增命令,先不改插件默认发现指引。

find 与 read/collect 的职责拆分、case_resource_path 共用、以及返回前用 reader 校验 hit,实现方向是清楚的。full-check / installed-package / pr-references 已通过。我没有额外的实现阻塞项,也没有逐条对照原始战役记录复核新加入的 findings,也没有重跑 PR 正文中的检索对照数字。请处理上述两点后再推新的 head。

@david6666666 david6666666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One compatibility issue with the documented discovery path.

entries. Add `--kinds case` when you specifically need a past observation.
Open a promising hit using its displayed `open` command or the JSON
`open_argv` argument list. If the result says `cannot open`, check the reason
and consult another hit when appropriate.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we keep the packaged discovery fallback here? On a host without the optional PyYAML extra, find 'ascend graph mode' --kinds skill ranks the skill but offers no open_argv: skill hits use checkout: and packaged projections are filtered out. Trying another skill hit won't help, while collect --relevance 'ascend graph mode' --limit 1 --material --json still returns the packaged skill on the same host. Please point this case to collect, or have find offer the corresponding packaged read command.

@Dong1017 Dong1017 added area:knowledge Knowledge line: packs, skill library, knowledge admission kind:task Bounded delivery task with a delegated slice labels Nov 16, 2026
Use the existing case bodies to express the recorded decisions, limits, and follow-up checks. Keep the tests focused on which outcome is retrieved and the evidence it carries.

Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Agent-assisted-by: Codex / gpt-6.1-sol [effort=max] (author)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>

Copy link
Copy Markdown
Collaborator

审阅了自上次评论以来的增量:80ec573eb5b635a938cc6e215f1fd2dbed8c0d7f → 当前 head 791cf4d00d416357acc9a3095e2813a88e0f36a7(2 个提交:1431037fe138679a0b225e1501dbbc43f97a2150 重写 packaged findings,791cf4d00d416357acc9a3095e2813a88e0f36a7 刷新 skill projections)。当前 head 仍不能 Approve。

上次的两点阻塞都还在:

  1. 必选检查 knowledge-model-policy 在该 head 上仍是 noncompliant。最新 declarations(run 43841803683)报 Material AI author 1: model is unapproved or suspended。PR 正文已把 head_sha 更新到当前 head,并把作者 3(Codex)写成 gpt-6.1-sol / max,先前作者 3 的 unknown 问题不再出现;作者 1(Claude Code earlier search implementation)的 model_id / effort 仍是 unknown。按 docs/knowledge-model-policy.md,未知值应如实填写,但不能通过准入。需要已批准模型且 effort 为 xhigh 或 max,或由批准人按规范给出 exception。请不要为了过检而改写并不确实知道的声明值。

  2. tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 仍是 status:proposed,产品决定仍是暂不把短指引推广为产品入口。这次增量没有改插件 / README / quickstart 的发现路径,plugins/inferrsi/skills/inferrsi/SKILL.md、README.md 和 docs/quickstart.md 仍把 find 写成默认入口。请先在 tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 上由请求者/维护者明确接受,或把交付收窄为新增命令、先不改默认发现指引。

另有两点请在这个 head 上处理或更新:

  1. @david6666666 在当前 head 上的行内评论(4583888248)仍然成立:这次增量没有改 knowledge_find.py。技能 hit 仍使用 checkout: 身份,packaged-skill 投影仍被跳过;没有可选 PyYAML 时,命中会被标为不可读且不提供 open_argv。请按该评论补上 packaged collect / read 回退,或在插件指引里明确这条路径。

  2. 正文「实测结果」里,第三行仍写在 80ec573e 上复跑得到 7/9。这次 findings 重写会改变检索词面,可能使该数字以及未标明测量 head 的「重写后」数字失效。请在当前 head 上复跑并更新表,或标明哪些数字仍只对旧 head 有效。

full-check(3.10/3.11)、installed-package 和 pr-references 已通过。我没有逐条对照原始战役记录复核这次重写的 findings,也没有重跑检索对照。

Use the packaged projection and its revision to resolve a Skill that the base installation cannot open from find.

Agent-assisted-by: Codex (author)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR review verdict — DO NOT PASS

Reviewed base 016c7c7df023944f0f46d8e777d6d31f1df1764c and head 866fd2aa2816652976c107d6ab09eb80d0343ef1.
Re-review after docs push that adds the packaged-skill PyYAML fallback guidance. Independent of author jiaran-king.

Blockers

  1. knowledge-model-policy still noncompliant on this head. declarations failed with Material AI author 1: model is unapproved or suspended (run 43846776491). PR-body author 1 still has model_id/effort = unknown. Keep unknown values truthful; obtain an approval-owner exception per docs/knowledge-model-policy.md, or revise protected-path authorship to qualifying declarations. Do not invent model ids.

  2. Default discovery promotion exceeds accepted #379 scope. #379 is still status:proposed and keeps the product decision: do not promote the #380 short guidance to the product entry; temporarily keep the existing catalogue and rg. This head still makes find → read the plugin/README/quickstart default (plugins/inferrsi/skills/inferrsi/SKILL.md, README.md, docs/quickstart.md). Expected: requester/maintainer acceptance on #379 for find as default, or narrow this PR to additive find/read --omit-resources without replacing the default discovery path.

  3. Branch is CONFLICTING vs current main. mergeable=false, mergeable_state=dirty. Conflicting paths include at least plugins/inferrsi/skills/inferrsi/scripts/inferrsi.py, tests/check_installed.py, and tests/test_knowledge_cli.py. Rebase/resolve onto current main before any Approve/merge.

Not blocking / observed

  • David's packaged fallback note (discussion_r4583888248) looks addressed in the plugin skill text on this head (collect --relevance → resource when PyYAML is missing). Keep that path through rebase.
  • Ranking implementation direction remains sound; ranking-quality table numbers were not re-run here.
  • No merge while dirty or while blockers 1–2 remain.

Checked / not checked

Checked: PR body, #379 accepted wording, declarations failure log, conflict paths via merge-tree, plugin SKILL fallback section, prior Dong/Amy/David findings. Did not re-run the labelled-query suites or full pytest on this conflicting head.

@david6666666 david6666666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One follow-up on the documented PyYAML fallback.

`open_argv` argument list.

When a Skill hit is unreadable because PyYAML is missing, use
`collect --relevance "<same query>" --limit 1 --material --json` to look up

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the fallback verify that it found the same skill? Without PyYAML, find 'cuda graph warmup' --kinds skill --limit 1 selects kernel-writing-guide, but the documented collect --relevance command selects ascend-aclgraph-capture-replay. Please match skill_documents.original to the selected hit's path, or describe this as a fresh search rather than opening the selected hit.

Copy link
Copy Markdown
Collaborator

审阅了自上次评论以来的增量:791cf4d00d416357acc9a3095e2813a88e0f36a7 → 当前 head 866fd2aa2816652976c107d6ab09eb80d0343ef1(1 个提交:docs: explain the packaged skill fallback without PyYAML,只改了 plugins/inferrsi/skills/inferrsi/SKILL.md 和 docs/operations.md)。当前 head 仍不能 Approve。

上次的两点阻塞都还在:

  1. 必选检查 knowledge-model-policy 在该 head 上仍是 noncompliant。最新 declarations(run 43846776491)报 Material AI author 1: model is unapproved or suspended。PR 正文已把 head_sha 更新到当前 head,但作者 1(Claude Code earlier search implementation)的 model_id / effort 仍是 unknown。按 docs/knowledge-model-policy.md,未知值应如实填写,但不能通过准入。需要已批准模型且 effort 为 xhigh 或 max,或由批准人按规范给出 exception。请不要为了过检而改写并不确实知道的声明值。

  2. tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 仍是 status:proposed。作者在 tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 评论 6913527304 中建议接受把 find 作为默认入口,但请求者/维护者尚未在 tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 记录决定,产品正文仍写“暂不将 task: 改进真实优化任务中的材料发现与使用 #380 所测短指引推广到产品入口”。这次增量只补充了无 PyYAML 时的回退说明,plugins/inferrsi/skills/inferrsi/SKILL.md、README.md 和 docs/quickstart.md 仍把 find 写成默认入口。请先在 tracking: Skill Indexer 检索效率、准确性与逐轮采用 #379 上得到明确接受,或把交付收窄为新增命令、先不改默认发现指引。

关于上次第 3 点:本次文档补充了 collect --relevance → resource 路径,但还不能算已解决。@david6666666 在当前 head 的行内评论(4584520972)指出,该回退按同一查询重新做 collect --relevance,不一定打开 find 选中的那条 skill;例子是 find 'cuda graph warmup' --kinds skill --limit 1 选中 kernel-writing-guide,而文档中的 collect --relevance 会选到 ascend-aclgraph-capture-replay。我没有在本机复跑这个例子,但文档把 collect --relevance "<same query>" 写成“查找该 packaged skill”,而该命令本身是一次新的相关性检索,确实不能保证对应同一条 hit。请按选中 hit 的路径去匹配 skill_documents.original,或把文档改写成一次新的搜索,而不是打开当前命中项。

上次第 4 点仍然存在:正文「实测结果」第三行仍写在 80ec573e 上复跑得到 7/9。本次文档增量没有更新该表,也未标明哪些数字仍只对旧 head 有效。请在当前 head 上复跑并更新,或明确标注测量 head。

当前 head 上 cursor[bot] 的 review(5918404550)为 CHANGES_REQUESTED / DO NOT PASS,其中知识模型策略与 #379 范围两点与上述 1、2 一致。本次增量是文档,不涉及新的精度或性能对比实验。我没有重跑检索对照,也没有在本机验证无 PyYAML 回退命令。

@jiangkuaixue123 jiangkuaixue123 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR review verdict — DO NOT PASS

Reviewed target main, PR base 016c7c7df023944f0f46d8e777d6d31f1df1764c and head 866fd2aa2816652976c107d6ab09eb80d0343ef1. Also checked integration against fetched main at 6c7e237935a22cc5e90e954d4f397376e753e7d8.

Delegation: the user directly requested an independent review of #466 and authorized posting review findings. Acting reviewer: Codex under @jiangkuaixue123 (repository write access verified); scope: code, tests and merge readiness; permitted action: publish this revision-bound Request changes review. This reviewer did not author or materially contribute to this PR.

当前不建议合并。以下是对已有意见的独立复核,不把重复问题计作新发现。

  1. [P2] 无 PyYAML 时的回退不能保证打开原来命中的 Skill。 已在当前 head、Python 3.11.15 禁用 site packages 的环境复现:find 'cuda graph warmup' --kinds skill --limit 1 --json 选中 kernel-writing-guide,路径为 methods/kernel-writing-guide/SKILL.md;而按 plugins/inferrsi/skills/inferrsi/SKILL.md:129-133 和 docs/operations.md:246-250 执行 collect --relevance 'cuda graph warmup' --limit 1 --material --json,返回的是 inferrsi.ascend.skills.ascend_aclgraph_capture_replay。两个命令都成功退出,但材料已经换了。请通过选中 hit 的身份/路径确定对应 package projection,并校验 skill_documents.original;或明确把回退描述为重新搜索、要求重新判断结果,不能称作读回该命中项。建议用这个例子补回归覆盖。确认 David 的现有意见 仍然成立。

  2. 必选 knowledge-model-policy 检查仍失败。 已读取当前 head 的最新 declarations 日志,明确报告 Material AI author 1: model is unapproved or suspended;也重新读取了 main 分支规则,确认该 context 仍是必选项。请保留真实的 unknown 声明,按仓库流程取得有权批准者的 exception,或提供符合规则的实际作者声明;普通代码 review 不能替代此项。

  3. 默认入口切换仍缺少记录下来的范围决定。 已读取 #379 正文及全部四条评论;最新提议 仍是在请求请求者/维护者接受 find 作为默认入口,未见接受记录。这不表示 #380 对旧短指引的负结果证明新工具无效;问题是本 PR 将 README、quickstart 和插件默认指引切到 find → read,而这项产品决定尚未记录。请在 #379 确认范围,或先保留原默认入口、把新命令作为增量能力交付。

  4. 当前分支存在合并冲突。 GitHub 返回 mergeable=false / mergeable_state=dirty;本地 git merge-tree --write-tree HEAD origin/main 在上述 fetched main 上也退出 1。当前实际冲突文件是 src/inferrsi/knowledge/data/repo_packs/inferrsi/knowledge/_provenance.json。请结合最新 main 重新生成一致的索引/provenance,处理冲突后在新 head 上重跑检查并请求复审。此处的冲突路径是本次复核结果,不沿用较早评审列出的路径。

验证:

  • Python 3.11.15 / macOS,独立 detached worktree。运行 knowledge find/CLI/operations、skill catalog/index、Codex/WorkSwarm/DSH 插件及 module boundaries 共九个测试文件:217 passed, 2 skipped, 1 failed。
  • 唯一失败为 tests/test_dsh_plugin.py::test_remote_ssh_connection_and_scoped_operations,其 Node 检查返回 AssertionError ERR_ASSERTION。在单独的 base worktree 运行相同 tests/check_dsh_remote.cjs 也失败。因此不归因为本 PR 新增回归,但本次套件不是全绿。
  • ruff check src tests、git diff --check 通过;tools/build_skill_index.py --source-revision 1431037fe138679a0b225e1501dbbc43f97a2150 --check 通过(80 metadata records)。
  • 本地 editable 安装的 SCM 自动版本推导受已有非版本 Git 标签影响;仅测试安装使用 SETUPTOOLS_SCM_PRETEND_VERSION=0.0.0,没有修改受审代码。上述结果不作为默认打包成功的证明。
  • 核对了新 findings 与仓库内原 Case 正文、检索评分、共享 reader/path 校验、精简读回和插件启动链;除上述已复现问题,未发现足够明确的新实现阻塞项。
  • 未运行全仓测试、Windows 实机、GPU/NPU 或真实 Agent 对照;未独立验证 PR 正文的优化收益/成本数据。本次是 PR 工程审查,不是 #379 的目标验收或 campaign PASS。

Move the find integration to the current Apps CLI and clarify that the no-PyYAML fallback searches packaged skills independently. Refresh the generated provenance in the next commit from this combined source revision.

Agent-assisted-by: Codex (author)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
…ression

Agent-assisted-by: Codex (author)
Signed-off-by: Zhou ziheng <jiaranran2@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:knowledge Knowledge line: packs, skill library, knowledge admission high priority kind:task Bounded delivery task with a delegated slice

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants