Skip to content

fix(record_run): refuse a host the declared profile does not describe (#55) - #56

Merged
shinaoka merged 1 commit into
mainfrom
fix/55-record-run-host-check
Nov 9, 2026
Merged

shinaoka merged 1 commit into
mainfrom
fix/55-record-run-host-check

Conversation

@shinaoka

@shinaoka shinaoka commented Nov 9, 2026

Copy link
Copy Markdown
Member

Closes #55.

何を閉じるか

record_run は宣言されたプロファイルとハードウェアが一致しないホストの記録を受け入れていた(#55: EPYC 7702P・SMT off を zen2-cpu 128T として記録)。記録を書く前に、宣言プロファイルの socket 数・SMT・モデルとホストを突き合わせ、一致しなければ拒否する。

変更

  • scripts/record_run.py: campaign declarations からのプロファイル解決、SMT は宣言時のみ比較、sibling map と引数(profile 名 2 つ)の拒否、notsupported を off と読まない。
  • benchmarks/suites/faer-kernels.yaml / faer-kernels-large.yaml: zen5-cpu に smt {2, offset 12} を宣言(説明文と共有 toolkit profile が既に書いている事実)。
  • tests/test_record_run_host_check.py: 新規 10 tests。
  • docs/timing-policy.md / result/INDEX.md: 追随。

検証(実測)

python -m unittest discover -s tests                    -> Ran 25 tests ... OK(新規 10 passed を含む)
python -m campaign_toolkit.declarations                 -> declarations ok: 2 suite(s)
python scripts/gen_index.py --check                     -> exit 0
campaign-check-generated --base origin/main --no-index   -> checked 25 pages, 0 regenerated
campaign-sync-pins --check --repo faer-benchmark        -> ok / ok

検証できていないこと

  • cargo test は origin/main でも通らない(backend.rs 21 件 + op.rs 1 件の既存エラー)。変更前後でエラー集合は同一・main.rs に新規エラーなし、までを確認したが、Rust 単体テストは未実行で、CSV 列も実バイナリでは見ていない。
  • 「一致ホストで実際に記録される」統合テストは作っていない(build 不可・CI に checkout 無し)。実機で通過を確認した。
  • profile の型不正ガードは足していない(declarations が CI で検証し、落ちれば書く前に止まる)。
  • broccoli の python は 3.8 のみ(toolkit は >=3.10)なので、上のデモは toolkit ソースを PYTHONPATH で 3.8 から import して実行した。

レビュー

GPT-6.1-sol の敵対的レビュー 2 ラウンド。1 ラウンド目 BLOCKER(SMT 未宣言を "off" と解釈して zen5-cpu の記録を全拒否)→ 修正。2 ラウンド目(README の言い過ぎ、args_from_env への配線、64T 行)→ 修正。

…#55)

`campaign_toolkit.record_run` compares a profile with the CPU sets and the NUMA
layout it declares, but never with the machine it runs on. faer#330's acceptance
run on the 64-logical SMT-off host was therefore recorded under `zen2-cpu` (128
logical, SMT on); `benchmarks/scripts/idle_cpus.py check --profile zen2-cpu` was
the only tool that noticed (exit 4). The write path was the permissive one.

* `scripts/record_run.py` refuses the host before the recorder builds, times or
  writes anything. It reads the host's online logical CPUs, its SMT state
  (`smt/active`, else the sibling map) and its sibling map, compares them with
  the profile's declaration and prints both sides (`declared 128 logical / smt on,
  measured 64 logical / smt off`). A profile that declares no `smt` block is
  compared on the CPU count only; a host with no sysfs is refused, not assumed;
  arguments that name two profiles are refused rather than guessed, because an
  option value that matches a profile name would otherwise be checked in place of
  the profile the run records under. The check lives in the campaign's entry
  point because the shared package is pinned by revision in `requirements.txt`,
  so a package fix could not be pinned here without merging it first.
* `benchmarks/profiles.yaml` gains `zen2-64-smt-off` - EPYC 7702P, 64 logical,
  SMT off, with the facts measured on the host (16 L3 domains of 4 CPUs, one NUMA
  node, boost 0, governor performance). `zen2-cpu` is unchanged: it declares the
  same server with SMT on, and one identifier must not name two machines.
  `zen5-cpu` gains the `smt` block its own description and the toolkit's shared
  profile already state, so the development host stays checkable instead of being
  refused by the new rule.
* `fbench` writes a `profile` column into every CSV row, taken from
  `FBENCH_PROFILE` (set by the entry point) in `args_from_env`, so a CSV names the
  machine it came from without the campaign directory around it.
* tests: a matching host is accepted and hands the profile to the recorder, a
  mismatch exits non-zero with both values and leaves no run directory, a
  differing sibling map is refused, a host with no topology is refused, two
  profile names in the arguments are refused, and the sibling map is the fallback
  where the kernel exposes no SMT switch.

Evidence on broccoli_vpn: `zen2-cpu` exits 1 with the message above and writes
nothing; `zen2-64-smt-off` passes the host check and reaches the recorder's next
gate. `cargo test` cannot build this crate at origin/main either (21 pre-existing
errors in `backend.rs`/`op.rs` against both the pinned 0.28.0 and 0.32.0
checkouts); the same error set comes back with this change, none in `main.rs`.
@shinaoka
shinaoka merged commit 01a1cb8 into main Nov 9, 2026
2 checks passed
@shinaoka
shinaoka deleted the fix/55-record-run-host-check branch November 9, 2026 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

record_run accepts a host that does not match the declared profile (EPYC 7702P SMT-off recorded as zen2-cpu 128T)

1 participant