Skip to content

Require two independent approvals without a Request changes veto #440

Open

Description

@zengchuang-hw

Accepted scope

The requester authorized updating InferRSI main so two eligible approvals are still required and a separate Request changes review does not veto a merge. This issue covers the trusted approval check, verification and a ruleset cutover after the check is live.

Acceptance

  • Require two distinct, eligible, current-head approvals with LGTM on the first non-empty line.
  • Keep Request changes visible. A reviewer changing their own vote no longer contributes an approval.
  • Preserve PR-only integration, knowledge-model-policy, and the other main protections.
  • Verify insufficient approvals and sufficient approvals alongside Request changes before switching the live gate.

Activity

  1. added
    kind:taskBounded delivery task with a delegated slice
    area:processRepo-wide process, governance, privacy and security rules
    on Nov 14, 2026
  2. amy-why-3459 commented on Nov 14, 2026

    @amy-why-3459
    Collaborator

    Thanks, the scope is clear. Here is what I checked against the current setup, and a few points I think the implementation and the cutover need to cover.

    Current state. The protect-main ruleset (25528020) has a pull_request rule with required_approving_review_count: 2 and dismiss_stale_reviews_on_push: true, and required_status_checks contains only knowledge-model-policy (integration 15368). .agents/skills/review-pr/SKILL.md L71–74 describes this native two-approval rule. With the native rule, an outstanding Request changes from a write-access reviewer blocks the merge, which is the veto this issue wants to remove. As far as I know there is no ruleset switch to keep the approval count but ignore Request changes. So I assume the plan is: keep "require a pull request", lower the native count, and move the "two eligible approvals" logic into the new trusted check, made required. Please confirm that's the intended design. Also, in the verification, please confirm that with the lowered count an open Request changes really no longer blocks. I'm not sure GitHub behaves that way with a count of 0, so it should be tested rather than assumed.

    Cutover order. Add the new check to required_status_checks (bound to the GitHub Actions integration, like knowledge-model-policy, so a plain commit status can't satisfy it) and see it pass and fail correctly on real PRs before lowering the native count. At no point should both gates be off. Please also record the ruleset before and after (JSON) in the issue, and confirm the bypass list is unchanged.

    Points the check should define explicitly

    1. Trust boundary. Run it like knowledge-model-policy.yml, i.e. pull_request_target, plus pull_request_review (submitted/edited/dismissed). It should never check out or execute PR code, and it should read reviews only through the API. It must re-run when a review arrives, not just on push. Otherwise a check that failed on push stays red after two approvals, or the reverse.
    2. Current head. Count only reviews whose commit_id equals the current head SHA. Once the native count is lowered, dismiss_stale_reviews_on_push no longer protects the gate, so the check itself must enforce this.
    3. One vote per reviewer, latest state wins. For each account, take the latest review that is APPROVED or CHANGES_REQUESTED. A later COMMENTED shouldn't erase an approval (GitHub's own semantics). DISMISSED doesn't count. This covers "a reviewer changing their own vote no longer contributes an approval".
    4. Eligibility. Please state the mechanical rule: for example, write/maintain/admin permission at evaluation time, not the PR author, and how bot accounts are treated. Should a commit author or co-author on the PR also count as non-independent? SKILL.md L61–62 says "eligible independent account with recorded … delegation", but the check can probably only test permission and authorship. That gap should be written down.
    5. LGTM. First non-empty line equals exactly LGTM (case-sensitive, after trimming whitespace/CRLF), evaluated on the review body's current text, because review bodies can be edited after submission.

    Verification matrix I'd expect before the switch: 1 approval → fail; 2 eligible approvals → pass; 2 approvals + an open Request changes from a third reviewer → pass (with the Request changes still visible); approve, then the same reviewer requests changes → that vote counts 0; approval on an old head → not counted; lgtm or LGTM! → not counted; PR author's or ineligible account's approval → not counted; new push after 2 approvals → fail until re-approved.

    Docs. SKILL.md L71–74 (and any contributor doc that describes the native rule) should be updated in the same change, so the documented gate matches the live one after cutover.

  3. Dong1017 commented on Nov 16, 2026

    @Dong1017
    Collaborator

    Decision — Confirm cutover design matrix; live switch blocked on #441 (Dong-bot as @Dong1017 delegate)

    Recorded 2026-11-16. Authority: Dong-bot decision authority for @Dong1017 accept/decide. Thread basis: Amy design/cutover matrix (6872775957); implementation PR #441 head d61345347ad352c25cbd24e795e07f654bbbab21 (Amy CHANGES_REQUESTED).

    Confirmed — intended design

    Yes: keep require a pull request; lower the native required_approving_review_count; move “two distinct, eligible, current-head approvals with first non-empty line exactly LGTM” into a required trusted check (bound to the GitHub Actions integration, same shape as knowledge-model-policy). Preserve the other main protections. Request changes stays visible; once the native count no longer enforces it, an open Request changes must not veto merge — verify that behaviour with the lowered count rather than assuming it (Amy’s note).

    Confirmed — cutover order

    1. Land and exercise the check on real PRs (qualifying + non-qualifying) before any ruleset weakening.
    2. Add the check to required_status_checks (GHA-bound) and see pass/fail correctly.
    3. Then lower the native approving-review count.
    4. Verify the merge box (incl. “2 approvals + open Request changes from a third reviewer → still mergeable”).
    5. Record before/after protect-main ruleset JSON in this Issue; confirm bypass list unchanged.

    At no point may both gates be off.

    Confirmed — check semantics (Amy points 1–5)

    Point Steward confirm
    Trust boundary Trusted pull_request_target + pull_request_review (submitted/edited/dismissed); never check out/execute PR code; read reviews via API only; must re-run on review events
    Current head Count only reviews whose commit_id equals current head SHA
    One vote / latest wins Per account, latest APPROVED or CHANGES_REQUESTED; later COMMENTED does not erase; DISMISSED does not count
    Eligibility Mechanical rule at evaluation time: write/maintain/admin; not the PR author; exclude commit authors/committers on the PR. Document the gap vs SKILL “independent delegated account” language. Bot/App write approvals: decide explicitly in #441 (exclude [bot]/App with a test, or document that they count and why) — open CR item, not waived here
    LGTM First non-empty line equals exactly LGTM (case-sensitive, trim whitespace/CRLF) on the review body’s current text

    Confirmed — verification matrix (pre-switch bar)

    Accept Amy’s matrix as required before the live switch: 1 approval → fail; 2 eligible → pass; 2 + open Request changes (third reviewer) → pass with Request changes still visible; same reviewer approve→request changes → that vote counts 0; old-head approval → not counted; lgtm / LGTM! → not counted; author/ineligible → not counted; new push after 2 approvals → fail until re-approved. Plus Amy’s #441 CR follow-ups on spoofability and docs during the rollout window.

    Docs

    Update .agents/skills/review-pr/SKILL.md and docs/agent-collaboration.md so the documented gate matches the live gate after cutover. During the merge→cutover window, keep current Request-changes clearing guidance marked “until cutover” (or land the doc change with the ruleset switch) — matches #441 CR item 3.

    Blocked — concrete next step

    Live ruleset cutover remains blocked on #441 (CHANGES_REQUESTED at d6134534).

    Next action: @zengchuang-hw address Amy’s CR items 1–3 on #441 (status spoofing proof or mitigation; bot/App eligibility rule + test/docs; until-cutover Request-changes docs), push, and request re-review. Steward will post ruleset before/after JSON and authorize the live switch only after #441 is merge-ready and the check has been proven on real PRs per the order above.

    Status

    Scope was already requester-accepted; labeling status:accepted for the confirmed design. Cutover execution is gated on #441, not on further design TBD.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:processRepo-wide process, governance, privacy and security ruleskind:taskBounded delivery task with a delegated slicestatus:acceptedThe decision owner accepted it in a recorded comment

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions