Skip to content

Automate triage reviews and add a LiveKit skill - #443

Merged
jserv merged 3 commits into
mainfrom
skills
Oct 23, 2026
Merged

jserv merged 3 commits into
mainfrom
skills

Conversation

@jserv

@jserv jserv commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

A bare /codetrial-triage now acts on the PRs it grades: a conflicting PR gets the fixed rebase text as a Request changes review, hygiene.sh posts one fixed-text Request changes review for the recurring requests maintainers type by hand (make hooks, make indent, merge commits, a fork's main, agent co-author trailers, CI regressions, missing screenshots), and each low or blocked PR gets a multi-agent review whose verified findings post-inline.sh posts inline. A request a maintainer already made at the current head is never repeated.

codetrial-livekit covers the Rust interviewer's side of LiveKit, and scripts/check-skill-routing.py checks that each request routes to the skill that owns it; the full query set routes 96 of 96 runs correctly.

Validated with scripts/test.sh portable, the skill-script tests (each new rule mutation-checked), and read-only sweeps over the open PRs.


Summary by cubic

A bare /codetrial-triage now acts on the PRs it grades instead of only listing them: a conflicting PR gets the fixed rebase text as a Request changes review, hygiene.sh posts one fixed-text Request changes review for the recurring requests maintainers type by hand, and each low or blocked PR gets a multi-agent review whose verified findings post-inline.sh posts inline. A request a maintainer already made at the current head is never repeated, including one made on a line of the head.

The sweep's reviewer runs with the maintainer's GitHub token nulled so a prompt injection in a contributor's diff cannot post or approve. A similar-work mirror is posted only to an open PR; a pair skipped because a head moved stays out of the ledger and is retried next sweep. The rebase recheck reads per reason: a conflicts entry needs CONFLICTING at the same head, a stale_base entry only the head.

New Features

  • codetrial-livekit covers the Rust interviewer's room loop, join tokens, agent dispatch, egress, and upgrading the LiveKit crates.
  • scripts/check-skill-routing.py verifies each request in scripts/skill-routing-queries.json routes to the skill that owns it; the full query set routes 96 of 96 runs correctly, and a run count below one or a launch failure fails plainly.
  • A failed hygiene.sh run is skipped and reported as unchecked, never posted, and a review's summary waits until every finding landed.

Written for commit ba97f82. Summary will update on new commits.

View guided diff Turn on auto-fix

cubic-dev-ai[bot]

This comment was marked as resolved.

cubic-dev-ai[bot]

This comment was marked as resolved.

jserv added 3 commits October 23, 2026 13:53
A bare /codetrial-triage now acts on the PRs it grades instead of only
listing them. A conflicting PR gets the fixed rebase text as a Request
changes review, so it shows as blocked until the author pushes.
hygiene.sh turns the requests maintainers keep typing by hand (make
hooks, make indent, merge commits, a fork's main, agent co-author
trailers, CI regressions, missing screenshots) into one fixed-text
review per head, and each low or blocked PR gets a multi-agent review
whose verified findings post-inline.sh posts inline.

Every post is pinned to the head it was read at, and nothing is asked
again that a maintainer already asked at that head, in a review, a
comment or on a line. Agy reads contributors' diffs sandboxed and
without the maintainer's GitHub token.
The interviewer is a Rust room participant that the web server
dispatches, not a LiveKit Agents SDK worker, so the lk agent tools
cannot drive it and agents keep reaching for them. codetrial-livekit
maps where rooms, tokens, dispatch, agent state, egress and the
transport patch live, how to look a LiveKit fact up for the pinned
crate, the connection checks, the rules room code follows, and how to
upgrade the crates.
A skill's description is all an agent reads before choosing it, so two
descriptions claiming the same request, or none claiming it, send work
to the wrong instructions with no error anywhere. check-skill-routing.py
runs each request in skill-routing-queries.json through claude -p in a
throwaway, credential-free copy of the tree and reports which skill
fired first. A run that fails, times out or never finishes counts as an
error, never as a near-miss that fired nothing.

It needs a logged-in claude, so it sits outside the gate;
codetrial-verify says when to run it.
@jserv
jserv merged commit 79c7abd into main Oct 23, 2026
10 checks passed
@jserv
jserv deleted the skills branch October 23, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant