Skip to content

[ENHANCEMENT] Mirror CodeRabbit pre-merge checks as a CI status #2164

Open

Description

@edelauna

Problem (one or two sentences)

CodeRabbit pre-merge check results exist only in its walkthrough comment. No CI status shows them, so no ruleset or merge queue rule can use them.

Context (who is affected and when)

Maintainers and authors see the results only if they open the comment. Evidence from this repository:

Desired behavior (conceptual, not technical)

A CI job reads the CodeRabbit walkthrough table and publishes one commit status that mirrors it.

  1. Triggers: issue_comment (created, edited) from coderabbitai[bot], pull_request_review (submitted, dismissed), pull_request (synchronize) and merge_group.
  2. Find the latest coderabbitai[bot] review.
    • If it is APPROVED and its commit_id equals the head SHA, publish success. CodeRabbit approval overrides the table. The maintainer decides when CodeRabbit approves, so no override label is needed.
    • Otherwise, go to step 3.
  3. Parse the table.
    • passed is success.
    • warning and inconclusive are neutral, with a warning annotation.
    • error is a failure.
  4. Publish neutral when there is no comment, the comment is older than the head commit, the parser finds no rows, or the CodeRabbit status says the review was skipped. The job must never fail because of a CodeRabbit problem.
  5. On merge_group, publish success at once. CodeRabbit does not comment on queue commits.
  6. Show the full table in the job summary, also when the approval override applies.

Rollout: start with error rows as neutral for two weeks. Then make error rows fail. Then add the status as a required check in the Protect Main ruleset. Do not use path filters on the workflow. Use if: on jobs instead.

Constraints / preferences (optional)

  • Use a separate workflow, not a new job in label-pr-review-state.yml. That file has 1,776 lines and the logic is inline in github-script closures. A separate workflow gives a stable required-check name.
  • Put the pure logic in a small CommonJS module, for example .github/scripts/coderabbit.js, with two functions: parsePreMergeTable and latestApprovalAtHead. Load it with require in github-script. Check out the default branch only, never PR code. Unit-test the functions by importing them, with the PR feat(mcp): add ToolRouter to the MCP marketplace #2125 comment as a fixture. Do not copy the YAML-extraction test pattern.
  • Publish a commit status, as updateReviewGate does. It needs statuses: write and no checks: write.
  • Treat the comment body as untrusted input. Do not put it in a shell command.
  • Do not write a second CodeRabbit comment parser if the existing one can be reused. Share only the login set (coderabbitai[bot], coderabbitai) now.

Open items:

Request checklist

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions