Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/reference-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,14 @@ the candidate mechanisms.
| [yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizhang/faster-cosyvoice/tree/f1049c59196c2c1a890079abc56056b43f27cd1d), inspected 2026-11-03; overall source rights UNKNOWN, selected file headers retain Apache-2.0 and unresolved inherited attribution | Bounded PCM client/server-credit, chunk planner, session/worker, Token2Wav and timing/quality-report inspection; [exact readset and per-file rights](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/faster-cosyvoice-source.json) | Extend [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) with [dependency-aware TTS playout](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/dependency-aware-tts-playout.md): route-origin credit versus client zero-buffer playback, same-session Mel dependencies, cumulative vocoding and partial-cost/token-derived denominators. Original analysis with immutable links only; candidate / empirical not_run / efficacy not_evaluated. Author reports remain separate; no copied implementation/media, new runtime or hardware/quality/service qualification. |
| [Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/156bed151826780e2ce0132f8d0e3540889ae8cd) + [paper v3](https://arxiv.org/abs/2607.02501v3), inspected 2026-11-04 (repository Apache-2.0; paper CC BY 4.0) | Selected fixed CPU-observation-to-complete-CPU-action boundary, client queue/profiler denominators, precision identity, per-task quality and source/artifact lineage; [exact readset and rights](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-source.json). Later SmolVLA reports remain separate from the August paper. | Extend registered owner `inference-benchmark-design` ([SKILL.md](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md)) with [VLA boundary and quality guidance](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/embodied-cpp-boundary-and-quality.md). Source-derived candidate/not_run; public sample arithmetic is distinct from episode recount and native execution. Generated actions are not control frequency; per-task loss, unequal precision, whole-device memory and unresolved measured-script/dirty-patch identity remain explicit. |

## GPU measurement validity and profiler attribution

General GPU measurement methodology for `inference-benchmark-design`, not an issue #60 task-selection source.

| Source / revision | Inspected scope | Reuse, extraction and limits |
| --- | --- | --- |
| NVIDIA NCU 2025.3, Nsight Systems 2025.3, CUDA 12.9.1/10.2 archives, hash-pinned SMI document; Triton v3.2.0 @ `9641643da6c52000c807b5eeed05edaec4402a67`; PyTorch 2.7; [robust-kbench paper v1](https://arxiv.org/html/2509.14279v1); [typed source inventory](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/metadata.json) | Selected clock/cache/replay, timing/graph, metric-name, flush, extension-identity and benchmark-exploit contracts, inspected 2026-11-15 | Extend `inference-benchmark-design` with [GPU measurement validity and profiler attribution](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md). Candidate / empirical not_run / efficacy not_evaluated; normalization/adoption is an unverified author proposal, four task-supplied cases lack raw evidence, and source review does not establish a gain or current-device metric availability. |

## SGLang versioned entry selection

The [three-pin responsibility comparison](../src/inferrsi/knowledge/data/skill_library/repositories/sglang/SKILL.md#choose-among-source-pins)
Expand Down
10 changes: 10 additions & 0 deletions docs/searchspace-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -654,6 +654,16 @@ remains unchanged; source use of GPU, TensorRT, Flow or HiFT does not alter
that boundary. Source-derived candidate / empirical not_run / efficacy
not_evaluated; author reports are not local quality or service results.

## GPU measurement validity and profiler attribution

[GPU measurement validity and profiler attribution](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/references/gpu-measurement-validity.md)
extends `inference-benchmark-design` with clock/cache controls, conditional
within-run attribution, work-count signatures and timing-trap checks. This is
evaluation guidance, not a hardware-resource optimization mechanism or added
numbered-layer coverage; the owner's N/A declaration remains applicable.
Candidate / empirical not_run / efficacy not_evaluated; task-supplied numerical
observations are unverified and establish no device or service gain.

## Embodied.cpp benchmark methodology

The [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md)
Expand Down
4 changes: 2 additions & 2 deletions docs/skill-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Generated from the registry, the skill sidecars and the two maintained
matrices. **Do not edit by hand**; regenerate after any registry, skill,
matrix or frozen-plan change.

- Input digest: `sha256:8f220788fd2d046f894d60991d7fa77c2b899705a88cd0970218f99fe73e276c`
- Input digest: `sha256:057218d1284e26df8db68c1177abd0a2ff73edbb175adc8441fe92a66a79e124`
- Regenerate: `python tools/build_skill_coverage.py`
- Verify without writing: `python tools/build_skill_coverage.py --check`

Expand Down Expand Up @@ -132,7 +132,7 @@ optimization is fully covered or that any procedure has measured benefit.
| [choose-parallel-training-plan](../src/inferrsi/knowledge/data/skill_library/methods/choose-parallel-training-plan/SKILL.md) | common | Compare stage, device-mesh and operator-sharding candidates for static distributed training while accounting for ... | Comparing stage, device-mesh and operator-sharding candidates for static distributed ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Alpa — [2201.12023v3](https://arxiv.org/abs/2201.12023v3)) | candidate · not_run · 1 readset(s) | no-eval |
| [dense-performance-evidence](../src/inferrsi/knowledge/data/skill_library/methods/dense-performance-evidence/SKILL.md) | common | Taxonomy of performance evidence an optimization agent needs, ordered by collection cost and diagnostic power. Maps ... | Deciding which performance evidence to collect next and which claim each evidence level ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Observe / Profile / Verify<br>2 RAM/Host · Verify<br>4 Network · Verify | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 0 readset(s) | — |
| [evaluate-elastic-serving-placement](../src/inferrsi/knowledge/data/skill_library/methods/evaluate-elastic-serving-placement/SKILL.md) | common | Compare migrating an LLM request between model replicas with changing its sequence-parallel execution group, including ... | Comparing migration of a running LLM request between compatible model replicas with ... | failure_modes, input_contract, produced_artifact, refusal_point<br>3 advice | 1 Accelerator · Diagnose / Configure / Optimize / Verify<br>2 RAM/Host · Diagnose / Configure / Optimize / Verify<br>4 Network · Diagnose / Configure / Optimize / Verify | 1 · Systems sources (Llumnix — [2406.03243v1](https://arxiv.org/abs/2406.03243v1)) | candidate · not_run · 1 readset(s) | no-eval |
| [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>8 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references ([Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc9804 ...)<br>2 · Methodology and corpus references ([Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/ ...)<br>Task-specific benchmark selection (issue #60) ([Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/15 ...)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...)<br>Task-specific benchmark selection (issue #60) ([yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizha ...) | candidate · not_run · 15 readset(s) | — |
| [inference-benchmark-design](../src/inferrsi/knowledge/data/skill_library/methods/inference-benchmark-design/SKILL.md) | common | Select task quality gates, reference parity checks and performance workloads for text, agentic coding, work agents ... | Selecting task quality gates, reference parity checks and performance workloads for text ... | failure_modes, input_contract, produced_artifact, refusal_point<br>9 advice | N/A: SKILL.md §§ Select the task and its evaluation contract, Statistical requirements, Shared-machine measurement discipline, and Search integrity and final-state verification design comparable evaluation; hardware identity and allocation isolation are comparison controls, not a hardware-resource mechanism. | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references ([Neroued/ninfer](https://github.com/Neroued/ninfer/tree/1882c6adbc9804 ...)<br>2 · Methodology and corpus references ([Xe-Forge repo + paper v1](https://github.com/IntelLabs/Xe-Forge/tree/ ...)<br>GPU measurement validity and profiler attribution (NVIDIA NCU 2025.3, Nsight Systems 2025.3, CUDA 12.9.1/10.2 archives ...)<br>Task-specific benchmark selection (issue #60) ([Embodied.cpp repo](https://github.com/SEU-PAISys/Embodied.cpp/tree/15 ...)<br>Task-specific benchmark selection (issue #60) ([Issue #60 report](https://github.com/InferMatrixLab/InferRSI/issues/6 ...)<br>Task-specific benchmark selection (issue #60) ([NVIDIA/NeMo-Speech.cpp @ 20999cc9](https://github.com/NVIDIA/NeMo-Spe ...)<br>Task-specific benchmark selection (issue #60) ([yuekaizhang/faster-cosyvoice @ f1049c59](https://github.com/yuekaizha ...) | candidate · not_run · 16 readset(s) | — |
| [inference-optimization-casebook](../src/inferrsi/knowledge/data/skill_library/methods/inference-optimization-casebook/SKILL.md) | common | How to structure and use a casebook of past optimization decisions as a decision-support tool. Extracts the casebook ... | Structuring and querying a casebook of past optimization decisions as decision support ... | failure_modes, input_contract, produced_artifact, refusal_point<br>4 advice | 1 Accelerator · Optimize | 1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd) | candidate · not_run · 0 readset(s) | no-eval |
| [kernel-writing-guide](../src/inferrsi/knowledge/data/skill_library/methods/kernel-writing-guide/SKILL.md) | common | Decision guide for writing custom inference kernels: when to write one, which framework to choose (Triton, CuTe DSL ... | Deciding whether to write a custom inference kernel, routing a kernel pattern to Triton ... | failure_modes, input_contract, produced_artifact, refusal_point<br>6 advice | 1 Accelerator · Optimize / Diagnose / Configure / Verify | 1 · Systems sources (Dao-AILab/causal-conv1d @ `cd81f0413cad2fc1e6f17e785ac39f59aae690cd` ...)<br>1 · Systems sources (NVIDIA/TensorRT-LLM @ f33e0e5fd)<br>1 · Systems sources (huggingface/transformers @ `0dad7b822255a0ae261ec45ae937371e859ffd1a` ...)<br>1 · Systems sources (mindspore-ai/akg @ `5aa15f3fe4d856eb60cd3b27447e04f5f1fb4dff` ...)<br>1 · Systems sources (sgl-project/sglang @ `16176278d0180f1a1a578e53e6eef432577876c2` ...)<br>1 · Systems sources (vllm-project/vllm @ `537a79f0fc5ba564d2bef43d54915b2f80f9f77d` (root ...) | candidate · not_run · 7 readset(s) | no-eval |
| [optimization-search-policy](../src/inferrsi/knowledge/data/skill_library/methods/optimization-search-policy/SKILL.md) | common | Policy for how an optimization agent should select, order, and stop optimization experiments. Covers candidate ... | Selecting, ordering and stopping optimization experiments: candidate generation ... | failure_modes, input_contract, produced_artifact, refusal_point<br>5 advice | 1 Accelerator · Optimize | 2 · Methodology and corpus references (JiusiServe/vllm-evolve @ 43cc82c)<br>2 · Methodology and corpus references (RRSI — arXiv 2609.24972v3)<br>2 · Methodology and corpus references (arXiv 2608.01867 (CRISP))<br>2 · Methodology and corpus references (microsoft/SkillOpt @ 343db22) | candidate · not_run · 9 readset(s) | no-eval |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"schema_version": 1,
"repository": "https://github.com/InferMatrixLab/InferRSI",
"source_revision": "0143bc4e0228866a07cdc6c94e11fe0d76b93ab2",
"source_revision": "ee8c924f61f0a8f17338b8b25909a410398067ad",
"registry": {
"path": "src/inferrsi/knowledge/data/skill_library/skill-index.json",
"sha256": "7904275c3383380db179287171e1430617aaa9ac4f439291eff8a71426987fbb"
Expand Down
Loading
Loading