Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
148 changes: 148 additions & 0 deletions .github/workflows/ci-ready.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,3 +98,151 @@ jobs:
env:
PR_BODY: ${{ github.event.pull_request.body }}
run: python3 tools/check_pr_references.py

host-compatibility:
# GitHub-hosted Windows and macOS CPU evidence, separate from the Ubuntu jobs above:
# a pass on Linux is not a pass on these hosts. It runs only the eight plugin and DSH
# nodes that an earlier Windows run recorded as failing (63 passed, 8 failed), then an
# installed-wheel check. A pass here does not rewrite that record. No GPU, remote
# service or credential is used; the DSH node talks to the fake SSH server that its
# own check starts. One host at a time, one pytest process, no full suite.
strategy:
fail-fast: false
max-parallel: 1
matrix:
os: [windows-latest, macos-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 30
permissions:
contents: read
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- uses: actions/setup-node@v4
with:
node-version: "22"
- name: Install dev deps and the build frontend
run: python -m pip install -e ".[dev]" build
- name: Test the eight plugin and DSH host nodes
# Started from Python so both hosts receive identical node IDs; no shell rewrites
# "::" or "[...]". The checkout path is absolute and PYTHONPATH is not set.
shell: python
env:
RESULTS: ${{ runner.temp }}/host-compatibility-results.xml
run: |
import os
import shutil
import subprocess
import sys
import xml.etree.ElementTree as ET

nodes = [
"tests/test_codex_plugin.py::test_comparison_preview_execution_status_and_report[codex-fail-1-FAIL]",
"tests/test_codex_plugin.py::test_isolated_runtime_can_export_dashboard[workswarm]",
"tests/test_codex_plugin.py::test_comparison_preview_execution_status_and_report[workswarm-pass-0-PASS]",
"tests/test_codex_plugin.py::test_comparison_preview_execution_status_and_report[workswarm-fail-1-FAIL]",
"tests/test_codex_plugin.py::test_comparison_preview_execution_status_and_report[workswarm-missing-2-INCONCLUSIVE]",
"tests/test_codex_plugin.py::test_release_archive_is_installable_and_all_versions_match[codex]",
"tests/test_codex_plugin.py::test_release_archive_is_installable_and_all_versions_match[workswarm]",
"tests/test_dsh_plugin.py::test_remote_ssh_connection_and_scoped_operations",
]
# The DSH node skips without Node.js, and every node needs Git or Python.
for tool in ("node", "git"):
path = shutil.which(tool)
if path is None:
raise SystemExit(tool + " is required by the selected nodes")
subprocess.run([path, "--version"], check=True)
results = os.environ["RESULTS"]
command = [sys.executable, "-m", "pytest", "-vv", "-rs", "-o", "addopts=",
"--junitxml=" + results, *nodes]
status = subprocess.run(
command, cwd=os.environ["GITHUB_WORKSPACE"], check=False
).returncode
if status:
raise SystemExit(status)
# A skipped or deselected node is not a pass: all eight must have run.
suite = next(ET.parse(results).getroot().iter("testsuite"))
counts = {name: int(suite.get(name))
for name in ("tests", "errors", "failures", "skipped")}
print(counts, flush=True)
if counts != {"tests": len(nodes), "errors": 0, "failures": 0, "skipped": 0}:
raise SystemExit("every selected host node must run and pass")
- name: Check a wheel built from the sdist in a fresh environment outside the checkout
# Runs even if a node above failed, so both results are recorded; either failure
# still fails the job.
if: ${{ !cancelled() }}
shell: python
run: |
import os
import shutil
import subprocess
import sys
from pathlib import Path

workspace = Path(os.environ["GITHUB_WORKSPACE"]).resolve()
work = Path(os.environ["RUNNER_TEMP"]).resolve() / "host-wheel"
log = work.parent / "host-wheel.log"
assert work != workspace and workspace not in work.parents, work
work.mkdir()
# Children never see PYTHONPATH, and the checker compares with this absolute path.
environment = {key: value for key, value in os.environ.items() if key != "PYTHONPATH"}
environment["INFERRSI_CHECKOUT_RESOURCES"] = str(workspace / "src" / "inferrsi")


def run(*command, cwd=work):
command = [str(part) for part in command]
result = subprocess.run(
command, cwd=cwd, env=environment, capture_output=True, text=True,
encoding="utf-8", errors="replace", check=False,
)
text = "$ " + " ".join(command) + "\n" + result.stdout + result.stderr
print(text, flush=True)
with log.open("a", encoding="utf-8") as stream:
stream.write(text)
if result.returncode:
raise SystemExit(f"exit {result.returncode}: {command[0]}")
return result.stdout


# python -m build writes the sdist and builds the wheel from that sdist.
dist = work / "dist"
run(sys.executable, "-m", "build", "--outdir", dist, cwd=workspace)
sdists, wheels = sorted(dist.glob("*.tar.gz")), sorted(dist.glob("*.whl"))
assert len(sdists) == 1 and len(wheels) == 1, (sdists, wheels)
venv = work / "env"
run(sys.executable, "-m", "venv", venv)
scripts = venv / ("Scripts" if os.name == "nt" else "bin")
python = scripts / ("python.exe" if os.name == "nt" else "python")
console = scripts / ("inferrsi.exe" if os.name == "nt" else "inferrsi")
run(python, "-m", "pip", "install", f"{wheels[0]}[skills]")
assert console.is_file(), console
shutil.copyfile(workspace / "tests" / "check_installed.py", work / "check_installed.py")
run(python, "-I", "check_installed.py")
# The package under test is the installed wheel, never the source checkout.
location = Path(
run(python, "-I", "-c", "import inferrsi; print(inferrsi.__file__)").strip()
).resolve()
assert venv.resolve() in location.parents, location
assert workspace not in location.parents, location
installed = run(
python, "-I", "-c", "from importlib.metadata import version; print(version('inferrsi'))"
)
# The console script and python -m inferrsi answer with the same bytes.
for arguments in (["--version"], ["benchmarks", "list"]):
module = run(python, "-I", "-m", "inferrsi", *arguments)
assert module.strip() and module == run(console, *arguments), arguments
if arguments == ["--version"]:
assert module == installed, (module, installed)
- name: Retain host results
if: always()
uses: actions/upload-artifact@v4
with:
name: host-compatibility-${{ matrix.os }}
path: |
${{ runner.temp }}/host-compatibility-results.xml
${{ runner.temp }}/host-wheel.log
retention-days: 30
40 changes: 40 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,46 @@

### Changed

- The CLI moves into the Apps package and the two root CLI modules are removed
(Refs #408, A1), with no forwarding shell, `sys.modules` alias or import
fallback:
- `inferrsi.cli` → `inferrsi.apps.cli` (`src/inferrsi/cli.py` →
`src/inferrsi/apps/cli/__init__.py`), which exposes the canonical `main`
- `inferrsi.arguments` → `inferrsi.apps.cli.arguments`
(`src/inferrsi/arguments.py` → `src/inferrsi/apps/cli/arguments.py`), which
holds `parser`, `make_spec` and `device_ids`
- The `inferrsi` console script names `inferrsi.apps.cli:main`, and
`python -m inferrsi` delegates to the same function. Commands, flags, exit
codes and stdout/stderr are unchanged. The shared Codex and WorkSwarm
launcher and the DSH remote bootstrap import the canonical modules.
- Importing `inferrsi.cli` or `inferrsi.arguments` now raises
`ModuleNotFoundError`, in external Python code as well as in this
repository, and a pickle that names an object by either path no longer
loads. Callers import `main` from `inferrsi.apps.cli` and the parser
helpers from `inferrsi.apps.cli.arguments`. Test injection points move
with the code: `inferrsi.cli._knowledge_reader` and `inferrsi.cli.files`
are the same names on `inferrsi.apps.cli`. Frozen evidence is not
rewritten.
- The tracked package root shrinks from ten files to eight. Besides
`__init__.py` and `__main__.py`, the six left (`summary_chart.py`,
`benchmark_operations.py` and the four MindIE files) wait for A2 (#408) and
M1 (#452). The gate's `APPLICATION_ENTRIES` lists only
`benchmark_operations.py`, and `inferrsi.cli` and `inferrsi.arguments` join
`RETIRED_ROOT_MODULES`, so a root file that returns under either name
fails.
- The temporary neutral-leaf entry for root `arguments.py`
(`technical_leaf.temporary_importers`, `TEMPORARY_LEAF_IMPORTERS`) and its
`temporary leaf importers` blocker are removed, and nothing replaces them.
The moved parser reads `REPO_TYPES` through `inferrsi.knowledge.catalog`,
which binds the one tuple object from
`inferrsi.resources.repository_packs`. An Apps module that imports the leaf
itself still fails as a second importer. The permanent
`inferrsi.infra.inspection` rule and the empty `compatibility_pairs` are
unchanged, and the manifest keeps the removed entry in `revision_history`.
This route for `REPO_TYPES` needs architecture review and is not
self-approved.
- Isolation is still incomplete: the 12 Harness hits into Infra (R2, #412)
and the five runtime resource loaders (R3, #412) remain.
- Eleven legacy root modules are removed (Refs #412, R1). Their implementations
already lived in `inferrsi.knowledge` and `inferrsi.apps`. The root files are
deleted in the change that moves their last consumers, with no forwarding
Expand Down
18 changes: 10 additions & 8 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ Qwen-Image denoising and real-model Qwen3-TTS under
packages. `inferrsi benchmarks run <package> ...` launches the package's
`benchmark.py`. Task-specific orchestration, sampling and assessment live inside
each package. The task library owns its `index.json` registration. The existing
`benchmark/registry.py` discovers the resources. The existing `arguments.py`,
`cli.py` and `benchmark_operations.py` provide application assembly and launch.
`benchmark/registry.py` discovers the resources. The CLI package `apps/cli/`
and the existing `benchmark_operations.py` provide application assembly and launch.
Each task retains its own protocol and task-specific code.

The [current-Agent optimization session](optimization-benchmarks.md) assembles
Expand Down Expand Up @@ -158,7 +158,7 @@ the comparator; the engine AB/BA path retains its independent acceptance contrac

```text
src/inferrsi/
cli.py, arguments.py Existing CLI
apps/cli/ CLI entry `main` and its argument parser
benchmark_operations.py Resolve a task entry and launch it through Harness
benchmark/
registry.py Generic task resource discovery
Expand Down Expand Up @@ -271,15 +271,17 @@ examples/ Current Qwen runner, image validator and v4 adapter
tests/fixtures/legacy/ Frozen historical validation dependencies for regressions
```

The package root holds exactly ten tracked files at this stage: `__init__.py`,
`__main__.py`, `cli.py`, `arguments.py`, `benchmark_operations.py`,
`summary_chart.py`, `mindie_context.py`, `mindie_mcp.py`, `mindie_project.py`
and `_mindie_read_worker.py`. #412 R1 deleted eleven root modules with no
The package root holds exactly eight tracked files at this stage: `__init__.py`,
`__main__.py`, `benchmark_operations.py`, `summary_chart.py`,
`mindie_context.py`, `mindie_mcp.py`, `mindie_project.py` and
`_mindie_read_worker.py`. #408 A1 moved the CLI and its parser to `apps/cli/`
and deleted root `cli.py` and `arguments.py` with no forwarding shell or alias;
`__main__.py` and the console script call `inferrsi.apps.cli:main`. #412 R1 deleted eleven root modules with no
forwarding shell or alias: the seven Knowledge shells, whose code is in
`knowledge/`; the three old-run forwarders `evaluation.py`, `execution.py` and
`reporting.py`, whose code is in `apps/legacy/`; and `inspection.py`, whose
host binding is `apps/inspection.py`. The final two-file root (`__init__.py`
and `__main__.py`) waits for the A1/A2 (#408) and M1 (#452) moves.
and `__main__.py`) waits for the A2 (#408) and M1 (#452) moves.
[module-layout.md](module-layout.md#retired-root-modules) maps each retired
path to its replacement and lists what still keeps the isolation gate false.

Expand Down
8 changes: 4 additions & 4 deletions docs/contracts/current-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Permitted shared technical leaves are only:
- `inferrsi.contracts.json_v1` — the canonical encoder (D002);
- `inferrsi.contracts.runspec` and `inferrsi.contracts.claim` — the existing RunSpec and Verdict technical types, with only RunSpec re-exported by the package initializer;
- `inferrsi.contracts.evidence` — the existing evidence segment names and pure path rules; it reads and writes no filesystem state and is not re-exported by the package initializer;
- `inferrsi.resources.repository_packs` — the neutral raw pack-data reader and the existing static data enumeration (D020), formerly root `inferrsi._repo_pack_data`. Its parent initializer `resources/__init__.py` imports nothing. Outside Knowledge it has one permanent importer, `inferrsi.infra.inspection`, which binds only `repository_pack`; that exact rule is not a compatibility pair. Root `arguments.py` reads `REPO_TYPES` from it under a separate temporary entry that A1 (#408) removes.
- `inferrsi.resources.repository_packs` — the neutral raw pack-data reader and the existing static data enumeration (D020), formerly root `inferrsi._repo_pack_data`. Its parent initializer `resources/__init__.py` imports nothing. Outside Knowledge it has one permanent importer, `inferrsi.infra.inspection`, which binds only `repository_pack`; that exact rule is not a compatibility pair. A1 (#408) removed the separate temporary entry for root `arguments.py`: the moved parser `inferrsi.apps.cli.arguments` reads the same `REPO_TYPES` object through `inferrsi.knowledge.catalog` and does not import the leaf.

The neutral leaf carries no target selection, task, scoring, scheduling, or public business DTO, and imports none of K/H/B/Apps. It still finds the Harness-owned `pack.json` through `files("inferrsi").joinpath("harness", "data", "repo_packs", ...)`, so reading it never imports Harness. Creating the evidence directories is host I/O and stays in Apps, in `inferrsi.apps.evidence_layout`; that module is not a shared leaf. The list is exact: a `contracts.` or `resources.` prefix admits no other module.

Expand Down Expand Up @@ -143,7 +143,7 @@ Decision: see [`gate-policy-runstore-and-first-prs.md`](gate-policy-runstore-and

Decision: old entry points, arguments, `Path`, stdout, stderr, and old run 0/1/2 stay byte-for-byte compatible. New Apps naming/envelope/exit mapping is fixed by D022, but K01/H01 do not implement a CLI.

Current reading: this covers the CLI commands and the `inferrsi` and `python -m inferrsi` entry points. It no longer covers old root Python module paths. The revised delivery decision in #412, #451 and #408 retires those paths without aliases, in the change that moves their last consumers. R1 (#412) removed the old-run root modules `evaluation`, `execution` and `reporting` together with root `inspection`; the old run is `inferrsi.apps.legacy` and the host inspection binding is `inferrsi.apps.inspection`. Root `cli.py` and `arguments.py` stay in place until A1 (#408), and R1 changed only their imports.
Current reading: this covers the CLI commands and the `inferrsi` and `python -m inferrsi` entry points. It no longer covers old root Python module paths. The revised delivery decision in #412, #451 and #408 retires those paths without aliases, in the change that moves their last consumers. R1 (#412) removed the old-run root modules `evaluation`, `execution` and `reporting` together with root `inspection`; the old run is `inferrsi.apps.legacy` and the host inspection binding is `inferrsi.apps.inspection`. A1 (#408) then moved root `cli.py` and `arguments.py` to `inferrsi.apps.cli` and `inferrsi.apps.cli.arguments` and deleted both root files; the console script and `python -m inferrsi` call `inferrsi.apps.cli:main`.

## D014 — events remain local, not a new shared event schema

Expand All @@ -165,8 +165,8 @@ Current status after R1 (#412):

- The seven root leaves are deleted, so `compatibility_pairs` in [`../legacy-knowledge-manifest.json`](../legacy-knowledge-manifest.json) is empty. The nine pair records stay in its `revision_history`, unchanged, as the record of this decision.
- The former ninth pair, `infra.inspection` → `inferrsi.resources.repository_packs`, is a live import that no root deletion removes. It is now a permanent exact rule outside the allowlist: that one module binds the one name `repository_pack` with one plain module-level import. The leaf's bytes, SHA-256, allowed imports, shape and import-free parent initializer stay locked, and no `resources.` or `contracts.` prefix is allowed.
- Root `arguments.py` → the same leaf, for `REPO_TYPES`, is a separate temporary entry. It adds a blocker until A1 (#408) removes root `arguments.py`.
- `overall isolation_complete` is still false. That temporary entry, twelve `harness` → `inferrsi.infra` hits in six Harness modules (R2) and five declared runtime resource loaders (R3) remain. The package root holds ten tracked files; the final two-file root waits for A1/A2 (#408) and M1 (#452).
- Root `arguments.py` → the same leaf, for `REPO_TYPES`, was a separate temporary entry with its own blocker. A1 (#408) removed both with the root file, and no Apps importer of the leaf replaced them.
- `overall isolation_complete` is still false. A1 (#408) cleared that temporary entry; twelve `harness` → `inferrsi.infra` hits in six Harness modules (R2) and five declared runtime resource loaders (R3) remain. The package root holds eight tracked files; the final two-file root waits for A2 (#408) and M1 (#452).
- The permanent rule and that manifest revision require formal architecture review. The implementer does not self-approve them, and passing tests are not that review.

[`../module-layout.md`](../module-layout.md) has the exact rules and their negative controls.
Expand Down
Loading
Loading