Skip to content

fix(harness): enforce publish guard against unsafe commit history and tier bypass - #76

Merged
BotshareAI merged 2 commits into
harness/autopilot-2026-10-30from
review/harness-guard-enforcement-20261031
Nov 1, 2026
Merged

BotshareAI merged 2 commits into
harness/autopilot-2026-10-30from
review/harness-guard-enforcement-20261031

Conversation

@wikki26

@wikki26 wikki26 commented Oct 31, 2026 •

Copy link
Copy Markdown

Scope

Focused follow-up for the outstanding deterministic guard/publish issues in #70. Target branch: harness/autopilot-2026-10-30; this does not modify that branch directly.

Fixes

  • Parse changed paths with Git's NUL-delimited --name-status -z; use the exact path as an argument when reading a file's diff, rather than parsing Git-quoted patch headers.
  • Scan every commit that the publisher's original bundle would transfer, including forbidden tier C/D path edits and synthetic secret-like content added and later removed. Reject agent merge commits.
  • Promote edits that remove lines from existing tests out of tier A; such edits cannot qualify for automatic tier-A merge.
  • Make --max-tier an actual failure after all tier promotions, including dependency manifests.

Regression evidence

Added 7 focused negative/positive tests to the existing tests/test_harness.py: tab-containing protected names, A-to-B rejection, valid A/A and B/B ceilings, removing an assertion without removing the test function, synthetic secret add/remove, protected path add/remove, and an untracked skip subsequently corrected in commit history. Adjusted the prior two-commit skip test expectation to reject unsafe published history.

These changes are in CI guard tooling and tests only; no robot-control or safety runtime changes, workflow activation, secrets, or published agent branches.

Hosted evidence (31 October 2026)

  • Run 41203133657: dedicated quality/test passed with 369/369 executed and zero skipped, including all seven added guard regression tests. ROS verifier smoke and governance baseline also passed.
  • DCO passed on the feature branch. The generic reusable repository-quality / quality/test is optional/skipped here; the repository-local dedicated quality/test is the executed gate.
  • The original harness: autonomous improvement loop and decisions.yaml lifecycle (design, off until founder approval on #43) #70 remains a draft. This patch is for Alex and Mohammad to inspect and incorporate, not permission to activate the harness or auto-merge robot changes.

…ling

Use NUL-delimited Git paths, scrutinize every published commit for
protected edits and secret-like content, disallow weakened tests at
tier A, and reject changes beyond the issue's selected tier.

Signed-off-by: Arumuga Vignesh SR <129583467+wikki26@users.noreply.github.com>
Add negative fixtures for protected names, deleted assertions and
intermediate secret/protected commits; enforce a scoped tier ceiling.
Also assert an untracked skip in published history cannot be sanitized
just by fixing its final file content.

Signed-off-by: Arumuga Vignesh SR <129583467+wikki26@users.noreply.github.com>

@panthera-momagdii panthera-momagdii left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Apply the selected task's tier ceiling to every commit in the history the publisher transfers.

Reviewed #76 at head 6726eb294077709d4454171c657d70e93f749c2b against its parent #70 at 54bdd2b, including the full two-file patch, tools/harness_guard.py, harness_lib.py, policy tiers and prior #70 review findings. The NUL-delimited path parsing, existing-test tier promotion and add-then-remove secret/protected-path checks address the earlier examples. However, the new history loop checks C/D paths, secrets, skips and binaries, but does not reject a tier-B intermediate edit for a --max-tier A task. The new ceiling check at the end compares only the net-diff tier.

Reproduction (isolated Git history, no real secrets): from a base containing ros2/src/pkg/pkg/node.py, make signed commit 1 changing its return value (tier B); make signed commit 2 reverting the source file to its base contents and adding ros2/src/pkg/test/test_new.py containing def test_new(): assert True (tier A). The net base..HEAD diff contains only the new test (tier A), while git diff --name-status parent..commit on each transferred commit contains the tier-B source path. Under the current run() checks, --max-tier A is therefore not rejected; harness_publish.push_bundle() still publishes both commits. This violates the selected task's history/scope ceiling even though the final tree is tier A.

Please compute an effective tier ceiling over all published commits (including per-commit test-line removals/dependency-tier promotions), fail when any commit exceeds --max-tier, and add a two-commit source-edit/revert + tier-A-test regression. Preserve the strict C/D and secret-history rejection, human gates and draft-only rollout. This PR is stacked on #70; #70's separately reported stage-4 all-skipped/neutral verification gap also remains unresolved and is not cleared by this review.

Verification: I ran the two-commit history scenario in an isolated local Git repo and confirmed the net diff is test-only while both intermediate diffs modify node.py. I inspected, but did not execute, harness_guard.py itself. Hosted Repository quality run 41203133657, exact head: 369/369 checker tests passed, 0 skipped; this negative case is not covered. NOT RUN: actual harness publish, live dry run, workflow activation or robot tests. This review does not approve #70 or authorize harness activation.

@BotshareAI
BotshareAI merged commit 6726eb2 into harness/autopilot-2026-10-30 Nov 1, 2026
6 checks passed
BotshareAI added a commit that referenced this pull request Nov 1, 2026
Review of #76 (panthera-momagdii): the guard now computes a tier for
every commit the publisher transfers, with the same promotions as the
net diff (allowed tier D mission paths and dependency manifests to B,
new tests in tier C areas to A, removed lines of existing tests to B).
The effective tier is the highest of the net change and every commit,
and --max-tier fails on it. Regressions: source edit then revert plus a
new test, and a test weakened then restored.

Review of #70 (wikki26): the stage 4 merge gate requires the checks in
policy merge_gate.required_checks (default quality/test and
quality/pr-evidence) to run and succeed on the head SHA; skipped or
neutral runs never satisfy them, a skipped reusable duplicate is
allowed, any other failing or pending run blocks, and every commit
status (DCO, CLA) must be success. Regressions: all skipped, all
neutral, missing required verifier, required pending, optional
failure, mixed success with optional skips, statuses unread, pending
and failed. The App needs Commit statuses read for stage 4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DZnPEN4YoUKzMSoLmaZCAS
Signed-off-by: Alex Reznichenko <info@botshare.ai>
@BotshareAI
BotshareAI deleted the review/harness-guard-enforcement-20261031 branch November 1, 2026 16:18

Copy link
Copy Markdown
Contributor

Thanks @wikki26 and @panthera-momagdii. Carried into #70 together with the follow-up 008633c, which answers the P1 here: the guard now computes a tier for every commit the publisher transfers, with the same promotions as the net diff (allowed tier D mission paths and dependency manifests to B, new tests in tier C areas to A, removed lines of existing tests to B); the effective tier is the highest of the net change and every commit, and --max-tier fails on it. Regressions in tests/test_harness.py: source edit then revert plus a new test (net diff test-only, rejected at --max-tier A, accepted at B), and a test weakened then restored. Both fail on 6726eb2. Please re-review on #70.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants