Repository navigation
[ENHANCEMENT] Mirror CodeRabbit pre-merge checks as a CI status #2164
Copy link
Copy link
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Problem (one or two sentences)
CodeRabbit pre-merge check results exist only in its walkthrough comment. No CI status shows them, so no ruleset or merge queue rule can use them.
Context (who is affected and when)
Maintainers and authors see the results only if they open the comment. Evidence from this repository:
- The walkthrough comment has a
Pre-merge checkstable with 8 rows (4 built-in, 4 custom). Each row hasCheck name,StatusandExplanationcolumns. - The status is an image, not text. The names seen across 40 PRs are
pre-merge-checks-passed.svg,-warning.svg,-error.svgand-inconclusive.svg. - The explanation text is cut off at about 200 characters. The full text is in the
Full detailsblocks. - CodeRabbit edits one comment per PR. Its review state does not follow the table. PR feat(mcp): add ToolRouter to the MCP marketplace #2125 has 8 of 8 rows passed and a
CHANGES_REQUESTEDreview. PRs fix(auto-approval): extract nested shell substitutions for allowlist #2068, fix: enforce and persist workspace-scoped auto-approve command lists (#1939) #2056 and chore(deps): align @vitest/coverage-v8 and @vitest/ui to the vitest 4.1.11 peer line #2037 have anAPPROVEDreview on the head commit and still showerrorrows. - CodeRabbit already posts a
CodeRabbitcommit status. Its descriptions are "Review in progress", "Review completed", "Review approved" and "Review skipped: excluded by label configuration". It does not carry the table. - The
Protect Mainruleset has a merge queue and 6 required checks. No required check reads CodeRabbit output.
Desired behavior (conceptual, not technical)
A CI job reads the CodeRabbit walkthrough table and publishes one commit status that mirrors it.
- Triggers:
issue_comment(created, edited) fromcoderabbitai[bot],pull_request_review(submitted, dismissed),pull_request(synchronize) andmerge_group. - Find the latest
coderabbitai[bot]review.- If it is
APPROVEDand itscommit_idequals the head SHA, publish success. CodeRabbit approval overrides the table. The maintainer decides when CodeRabbit approves, so no override label is needed. - Otherwise, go to step 3.
- If it is
- Parse the table.
passedis success.warningandinconclusiveare neutral, with a warning annotation.erroris a failure.
- Publish neutral when there is no comment, the comment is older than the head commit, the parser finds no rows, or the
CodeRabbitstatus says the review was skipped. The job must never fail because of a CodeRabbit problem. - On
merge_group, publish success at once. CodeRabbit does not comment on queue commits. - Show the full table in the job summary, also when the approval override applies.
Rollout: start with error rows as neutral for two weeks. Then make error rows fail. Then add the status as a required check in the Protect Main ruleset. Do not use path filters on the workflow. Use if: on jobs instead.
Constraints / preferences (optional)
- Use a separate workflow, not a new job in
label-pr-review-state.yml. That file has 1,776 lines and the logic is inline ingithub-scriptclosures. A separate workflow gives a stable required-check name. - Put the pure logic in a small CommonJS module, for example
.github/scripts/coderabbit.js, with two functions:parsePreMergeTableandlatestApprovalAtHead. Load it withrequireingithub-script. Check out the default branch only, never PR code. Unit-test the functions by importing them, with the PR feat(mcp): add ToolRouter to the MCP marketplace #2125 comment as a fixture. Do not copy the YAML-extraction test pattern. - Publish a commit status, as
updateReviewGatedoes. It needsstatuses: writeand nochecks: write. - Treat the comment body as untrusted input. Do not put it in a shell command.
- Do not write a second CodeRabbit comment parser if the existing one can be reused. Share only the login set (
coderabbitai[bot],coderabbitai) now.
Open items:
- CodeRabbit does not edit its comment when it approves. In fix(auto-approval): extract nested shell substitutions for allowlist #2068 the comment was updated at 12:31 and the approval came at 13:13.
pull_request_reviewhas a read-only token on fork PRs, so fork approvals need the existing follow-up pattern (issues Fork PR review events leave labels stale until the hourly sweep #2046 and Label PR review state workflow is invalid: it listens to itself #2086). Check whetherlabel-pr-review-state-fork-review-followup.ymlcan be extended to dispatch the new workflow. - The table format and image names are not documented by CodeRabbit. The parser must fail soft.
- The
[IGNORED]row tag that the CodeRabbit docs describe is not used here. No example exists, so its format is unverified. - The CodeRabbit docs do not describe approval or re-review behavior. Everything above is observed behavior.
Request checklist
- I searched the existing Issues and Discussions for duplicates (nearest: chore: improve CodeRabbit review coverage for three common bug patterns #1497, which changes review instructions and does not add a CI mirror)
- This describes a specific problem with clear context and impact
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Activity