Skip to content
Merged
1 change: 1 addition & 0 deletions docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,7 @@ the candidate mechanisms.
| [Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/60#issuecomment-6310920198), 2026-10-21; eight Git source inventories in the [owner metadata](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json) | Task/quality/workload separation across text, agentic coding, work agents, TTS, full-duplex interaction and image/video; selected pinned AIPerf/realtime/FDB/video sources rechecked and the current InferRSI metric contract inspected | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [task selection](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/task-benchmark-selection.md) and [command templates](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/benchmark-commands.md). Candidate guidance; source review and bounded smoke do not qualify all tasks or establish a performance gain. Replay scores, reference parity and task quality retain separate meanings. |
| [NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Speech.cpp/tree/20999cc921d2b14aa58708f61756a8b2e787ba20), inspected 2026-11-01 | Selected cache-aware encoder/config, runner, SDK bounds, endpoint policy, geometry test/CMake and ASR benchmark; complete 14-file readset and Apache-2.0 / parakeet MIT attribution retained in the reference | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [cache-aware ASR stream boundaries](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/cache-aware-asr-stream-boundaries.md): first/steady mel geometry and decoded clock, stream-slot reset/reuse, EOU/EOF/reset and model-test SKIP boundaries. Source-derived candidate / empirical not_run; benchmark figures remain author-reported with precision, FE and trial limits. No model/device/quality/long-audio/load/service qualification. |
| [yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizhang/faster-cosyvoice/tree/f1049c59196c2c1a890079abc56056b43f27cd1d), inspected 2026-11-03; overall source rights UNKNOWN, selected file headers retain Apache-2.0 and unresolved inherited attribution | Bounded PCM client/server-credit, chunk planner, session/worker, Token2Wav and timing/quality-report inspection; [exact readset and per-file rights](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/faster-cosyvoice-source.json) | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [dependency-aware TTS playout](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/dependency-aware-tts-playout.md): route-origin credit versus client zero-buffer playback, same-session Mel dependencies, cumulative vocoding and partial-cost/token-derived denominators. Original analysis with immutable links only; candidate / empirical not_run / efficacy not_evaluated. Author reports remain separate; no copied implementation/media, new runtime or hardware/quality/service qualification. |
| [Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/156bed151826780e2ce0132f8d0e3540889ae8cd) + [paper v3](https://arxiv.org/abs/2607.02501v3), inspected 2026-11-04 (repository Apache-2.0; paper CC BY 4.0) | Selected fixed CPU-observation-to-complete-CPU-action boundary, client queue/profiler denominators, precision identity, per-task quality and source/artifact lineage; [exact readset and rights](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-source.json). Later SmolVLA reports remain separate from the August paper. | Extend registered owner `inference-benchmark-design` ([SKILL.md](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md)) with [VLA boundary and quality guidance](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md). Source-derived candidate/not_run; public sample arithmetic is distinct from episode recount and native execution. Generated actions are not control frequency; per-task loss, unequal precision, whole-device memory and unresolved measured-script/dirty-patch identity remain explicit. |

## SGLang versioned entry selection

Expand Down
13 changes: 13 additions & 0 deletions docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -653,3 +653,16 @@ hardware coverage or hardware prerequisite. The owner's existing N/A reason
remains unchanged; source use of GPU, TensorRT, Flow or HiFT does not alter
that boundary. Source-derived candidate / empirical not_run / efficacy
not_evaluated; author reports are not local quality or service results.

## Embodied.cpp benchmark methodology

The [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md)
method uses [Embodied.cpp at 156bed15](https://github.com/SEU-PAISys/Embodied.cpp/tree/156bed151826780e2ce0132f8d0e3540889ae8cd)
and [arXiv:2607.02501v3](https://arxiv.org/abs/2607.02501v3) as source context
for observation-to-action timing, generated versus replayed actions, precision
identity and per-task closed-loop quality. Its [operating reference](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md)
and [readset](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-source.json)
retain the later author's measurement and source-version limits.

These are comparison methods. Experiment hardware is source context rather
than a device prerequisite or a resource mechanism for this method.
4 changes: 2 additions & 2 deletions docs/skill-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Generated from the registry, the skill sidecars and the two maintained
matrices. **Do not edit by hand**; regenerate after any registry, skill,
matrix or frozen-plan change.

- Input digest: `sha256:bd877213543e995abb36ec11e88ef21be1df4374ce1dbdb2824e419bcc6fa64c`
- Input digest: `sha256:cd71660769696a01153a9bce7a7b2165d02ef359c405086827f7b65425718f77`
- Regenerate: `python tools/build_skill_coverage.py`
- Verify without writing: `python tools/build_skill_coverage.py --check`

Expand Down Expand Up @@ -132,7 +132,7 @@ optimization is fully covered or that any procedure has measured benefit.
| [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md) | common | Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for ... | Comparing stage, device-mesh and operator-sharding candidates for static distributed ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3)) | candidate · not_run · 1 readset(s) | no-eval |
| [dense-performance-evidence](../src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md) | common | Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps ... | Deciding which performance evidence to collect next and which claim each evidence level ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Observe / Profile / Verify<br>2 RAM/Host · Verify<br>4 Network · Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | — |
| [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | common | Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including ... | Comparing migration of a running LLM request between compatible model replicas with ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>2 RAM/Host · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1)) | candidate · not_run · 1 readset(s) | no-eval |
| [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>7 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references ([Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc9804 ...)<br>2 · Methodology and corpus references ([Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/ ...)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...)<br>Task-specific benchmark selection (issue #60) ([yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizha ...) | candidate · not_run · 13 readset(s) | — |
| [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>8 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references ([Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc9804 ...)<br>2 · Methodology and corpus references ([Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/ ...)<br>Task-specific benchmark selection (issue #60) ([Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/15 ...)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...)<br>Task-specific benchmark selection (issue #60) ([yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizha ...) | candidate · not_run · 15 readset(s) | — |
| [inference-optimization-casebook](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/SKILL.md) | common | How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook ... | Structuring and querying a casebook of past optimization decisions as decision support ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval |
| [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md) | common | Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL ... | Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton ... | failure_modes, input_contract, produced_artifact, refusal_point<br>6 advice | 1 Accelerator · Optimize / Diagnose / Configure / Verify | 1 · Systems sources (Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` ...)<br>1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` ...)<br>1 · Systems sources (mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` ...)<br>1 · Systems sources (sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` ...)<br>1 · Systems sources (vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root ...) | candidate · not_run · 7 readset(s) | no-eval |
| [optimization-search-policy](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | Selecting, ordering and stopping optimization experiments: candidate generation ... | failure_modes, input_contract, produced_artifact, refusal_point<br>5 advice | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)<br>2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 9 readset(s) | no-eval |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"schema_version": 1,
"repository": "https://github.com/InferMatrixLab/InferRSI",
"source_revision": "c8a0c20ab7cd6d161e9f347d42f84f436a8e9279",
"source_revision": "23d27be8bde499f8dadd1436657017cf2f4077ab",
"registry": {
"path": "src/inferrsi/knowledge/data/skill_library/skill-index.json",
"sha256": "7904275c3383380db179287171e1430617aaa9ac4f439291eff8a71426987fbb"
Expand Down
Loading
Loading