Repository navigation
fix(harness): enforce publish guard against unsafe commit history and tier bypass - #76
Conversation
…ling Use NUL-delimited Git paths, scrutinize every published commit for protected edits and secret-like content, disallow weakened tests at tier A, and reject changes beyond the issue's selected tier. Signed-off-by: Arumuga Vignesh SR <129583467+wikki26@users.noreply.github.com>
Add negative fixtures for protected names, deleted assertions and intermediate secret/protected commits; enforce a scoped tier ceiling. Also assert an untracked skip in published history cannot be sanitized just by fixing its final file content. Signed-off-by: Arumuga Vignesh SR <129583467+wikki26@users.noreply.github.com>
panthera-momagdii
left a comment
There was a problem hiding this comment.
[P1] Apply the selected task's tier ceiling to every commit in the history the publisher transfers.
Reviewed #76 at head 6726eb294077709d4454171c657d70e93f749c2b against its parent #70 at 54bdd2b, including the full two-file patch, tools/harness_guard.py, harness_lib.py, policy tiers and prior #70 review findings. The NUL-delimited path parsing, existing-test tier promotion and add-then-remove secret/protected-path checks address the earlier examples. However, the new history loop checks C/D paths, secrets, skips and binaries, but does not reject a tier-B intermediate edit for a --max-tier A task. The new ceiling check at the end compares only the net-diff tier.
Reproduction (isolated Git history, no real secrets): from a base containing ros2/src/pkg/pkg/node.py, make signed commit 1 changing its return value (tier B); make signed commit 2 reverting the source file to its base contents and adding ros2/src/pkg/test/test_new.py containing def test_new(): assert True (tier A). The net base..HEAD diff contains only the new test (tier A), while git diff --name-status parent..commit on each transferred commit contains the tier-B source path. Under the current run() checks, --max-tier A is therefore not rejected; harness_publish.push_bundle() still publishes both commits. This violates the selected task's history/scope ceiling even though the final tree is tier A.
Please compute an effective tier ceiling over all published commits (including per-commit test-line removals/dependency-tier promotions), fail when any commit exceeds --max-tier, and add a two-commit source-edit/revert + tier-A-test regression. Preserve the strict C/D and secret-history rejection, human gates and draft-only rollout. This PR is stacked on #70; #70's separately reported stage-4 all-skipped/neutral verification gap also remains unresolved and is not cleared by this review.
Verification: I ran the two-commit history scenario in an isolated local Git repo and confirmed the net diff is test-only while both intermediate diffs modify node.py. I inspected, but did not execute, harness_guard.py itself. Hosted Repository quality run 41203133657, exact head: 369/369 checker tests passed, 0 skipped; this negative case is not covered. NOT RUN: actual harness publish, live dry run, workflow activation or robot tests. This review does not approve #70 or authorize harness activation.
Review of #76 (panthera-momagdii): the guard now computes a tier for every commit the publisher transfers, with the same promotions as the net diff (allowed tier D mission paths and dependency manifests to B, new tests in tier C areas to A, removed lines of existing tests to B). The effective tier is the highest of the net change and every commit, and --max-tier fails on it. Regressions: source edit then revert plus a new test, and a test weakened then restored. Review of #70 (wikki26): the stage 4 merge gate requires the checks in policy merge_gate.required_checks (default quality/test and quality/pr-evidence) to run and succeed on the head SHA; skipped or neutral runs never satisfy them, a skipped reusable duplicate is allowed, any other failing or pending run blocks, and every commit status (DCO, CLA) must be success. Regressions: all skipped, all neutral, missing required verifier, required pending, optional failure, mixed success with optional skips, statuses unread, pending and failed. The App needs Commit statuses read for stage 4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DZnPEN4YoUKzMSoLmaZCAS Signed-off-by: Alex Reznichenko <info@botshare.ai>
|
Thanks @wikki26 and @panthera-momagdii. Carried into #70 together with the follow-up 008633c, which answers the P1 here: the guard now computes a tier for every commit the publisher transfers, with the same promotions as the net diff (allowed tier D mission paths and dependency manifests to B, new tests in tier C areas to A, removed lines of existing tests to B); the effective tier is the highest of the net change and every commit, and Generated by Claude Code |
Scope
Focused follow-up for the outstanding deterministic guard/publish issues in #70. Target branch:
harness/autopilot-2026-10-30; this does not modify that branch directly.Fixes
--name-status -z; use the exact path as an argument when reading a file's diff, rather than parsing Git-quoted patch headers.--max-tieran actual failure after all tier promotions, including dependency manifests.Regression evidence
Added 7 focused negative/positive tests to the existing
tests/test_harness.py: tab-containing protected names, A-to-B rejection, valid A/A and B/B ceilings, removing an assertion without removing the test function, synthetic secret add/remove, protected path add/remove, and an untracked skip subsequently corrected in commit history. Adjusted the prior two-commit skip test expectation to reject unsafe published history.These changes are in CI guard tooling and tests only; no robot-control or safety runtime changes, workflow activation, secrets, or published agent branches.
Hosted evidence (31 October 2026)
repository-quality / quality/testis optional/skipped here; the repository-local dedicatedquality/testis the executed gate.