Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"name": "codejury",
"description": "Code Jury: independent AI code reviewers and one judge, from inside Claude Code.",
"owner": {
"name": "agentsdance",
"url": "https://github.com/agentsdance"
},
"plugins": [
{
"name": "codejury",
"source": "./plugin",
"description": "Run a Code Jury review of a pull request, and the multi-agent review loop skill."
}
]
}
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

## Unreleased

- Code Jury is now a Claude Code plugin: `claude plugin marketplace add agentsdance/codejury`, then
`claude plugin install codejury@codejury`. `/codejury:review <pr-url>` runs a review from a Claude
Code session with the web console enabled and its link shown by default, without pushing unless
asked. A PR URL is required; Claude asks for it if omitted. Pass `--web=false` to disable the console.
The review-loop skill moved from `skill/SKILL.md` to `plugin/skills/multi-agent-code-review/SKILL.md`.
The manual guidance now uses the current
agent configuration and exact review session IDs (#115).
- Resolve offline local GitLab reviews from the caller's valid cached remote default
when available, instead of requiring a redundant `--trunk` override.
- Reject invalid or unavailable trunk branches before launching review agents.
Expand Down
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,26 @@ jury help review # complete review options
`jury <pr-url>` is shorthand for `jury review <pr-url>`. You can also run without installing:
`npx @agentsdance/codejury review <pr-url> --reviewer claude --push=false`.

### Claude Code plugin

Code Jury is also a [Claude Code plugin](https://code.claude.com/docs/en/plugins). Add this
repository as a plugin marketplace and install it:

```sh
claude plugin marketplace add agentsdance/codejury
claude plugin install codejury@codejury
```

In a session, `/codejury:review <pr-url>` runs `jury review` (or `npx @agentsdance/codejury`
when `jury` is not installed) with `--web=true` and, unless you pass `--push`, `--push=false`.
Claude shares the web console link, summarizes the findings and verdicts when the review ends,
and leaves the console running. Pass `--web=false` to disable it. The plugin requires a PR URL;
if omitted, Claude asks for it before starting. The plugin also ships the
`multi-agent-code-review` skill for running the review loop by hand. Agent CLIs are still installed
and authenticated separately. `--push=false` disables pushing; it does not make the review read-only.
Single-PR checkouts are temporary and their unpushed fixes are removed during cleanup.
See the [plugin guide](plugin/README.md) for prerequisites, updates, and local installation checks.

## Screenshots

A review of Code Jury PR #67, with Droid judging and Claude, Codex, and Grok reviewing.
Expand Down
13 changes: 13 additions & 0 deletions plugin/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"name": "codejury",
"displayName": "Code Jury",
"description": "Review a pull request with independent AI reviewers and one judge, using the Code Jury CLI.",
"author": {
"name": "agentsdance",
"url": "https://github.com/agentsdance"
},
"homepage": "https://github.com/agentsdance/codejury#readme",
"repository": "https://github.com/agentsdance/codejury",
"license": "MIT",
"keywords": ["code-review", "pull-request", "agents", "codex", "claude"]
}
78 changes: 78 additions & 0 deletions plugin/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Code Jury for Claude Code

Review PRs with independent AI code reviewers and a judge from a Claude Code session. The judge
triages findings and fixes accepted issues; Jury repeats the loop until the reviewers agree or the run
reaches its limit.

## Install

```sh
claude plugin marketplace add agentsdance/codejury
claude plugin install codejury@codejury
```

Start a new Claude Code session after installing, then run:

```text
/codejury:review https://github.com/owner/repo/pull/123
/codejury:review https://github.com/owner/repo/pull/123 --reviewer claude --judge codex --rounds 3
```

The first example uses your saved agent settings. The second overrides the reviewer, judge, and
round limit. Both open the web console and display its link in Claude Code.

A PR URL is required. If you run `/codejury:review` without one, Claude asks for the URL before
starting. Local-branch and local-directory reviews are outside this plugin's scope.
You can pass multiple related PR URLs or other `jury review` flags. With a PR URL, `--dir` selects
where checkouts and run records are stored. Resuming a run also requires its original PR URL(s).

You need Node.js 20.10 or newer, npm, Git, and installed, authenticated agent CLIs. GitHub PR URLs
also need an authenticated `gh` CLI. The plugin uses an installed `jury`, or
`npx -y @agentsdance/codejury` if `jury` is absent. It does not bundle or authenticate agent CLIs.
Use `jury agents` (or `npx -y @agentsdance/codejury agents`) to inspect your setup.

## Behavior

- `/codejury:review` runs only when invoked by the user. It respects your CLI flags,
`jury.config.json`, and saved agent/model defaults.
- It defaults to `--web=true`: the browser console opens and Claude shares its URL. Claude reports
the review result when it is ready and leaves the console running for you to inspect.
Pass `--web=false` for results only in Claude Code and a command that exits after review.
- It defaults to `--push=false`, so Jury does not push. The judge can still edit files and Jury
can create commits. Pass `--push` to publish them. Single-PR URL checkouts are temporary and
their unpushed fixes are removed during cleanup.
- Explicit `--web` and `--push` values take precedence.
- `multi-agent-code-review` provides guidance for manually coordinating reviewers and triaging
findings. Its source moved from `skill/SKILL.md` to `plugin/skills/multi-agent-code-review/SKILL.md`;
update any copies or symlinks that used the old path.

## Update

```sh
claude plugin marketplace update codejury
claude plugin update codejury@codejury
```

The plugin deliberately omits a version field so updates follow the repository without maintaining
a second release number. The installed Code Jury CLI is separate; update a global CLI installation
with `npm install -g @agentsdance/codejury` when needed.

## Develop and validate

From the repository root:

```sh
claude plugin validate ./plugin
claude plugin validate .
node --test test/plugin.test.js
claude --plugin-dir ./plugin
```

To test marketplace installation, add the local repository path as a marketplace and install
`codejury@codejury`. Use an isolated `CLAUDE_CONFIG_DIR` for development so this does not change your
normal marketplace settings. `claude plugin details codejury` should list `review` and
`multi-agent-code-review`.

The plugin lives under `plugin/` so installation includes only its manifest, skills, and this guide.
Skill references to repository documentation use GitHub URLs because the rest of the source tree
is not part of the installed plugin.
43 changes: 29 additions & 14 deletions skill/SKILL.md → ...n/skills/multi-agent-code-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,43 +5,56 @@ description: Review a pull request with several independent AI reviewers (codex,

# Multi-agent code review

This plugin supports PR reviews only. Require a PR URL before starting reviewers; if it is missing,
ask for it. Do not infer a local-branch or local-directory review from the current checkout.

You are the **main agent**. You write code, answer findings, and own the commit. Reviewers only read
and report — never let one edit files.

The CLI owns the mechanics; you own the judgement. See `DESIGN.md` for the split.
The CLI owns the mechanics; you own the judgement. See [DESIGN.md](https://github.com/agentsdance/codejury/blob/master/DESIGN.md) for the split.

For the automated loop, use `/codejury:review`; the configured judge owns its fixes. The guidance
below is for explicitly requested manual review and triage. Do not edit a checkout while an
automated Jury run is using it. In CLI examples, substitute `npx -y @agentsdance/codejury` for
`jury` if it is not installed, and keep the same `--dir` and `--run` when working with saved findings.

## The loop

```
round N
├─ worktree at the pushed HEAD
├─ worktree at the commit being reviewed
├─ launch every reviewer in parallel, in the background
├─ triage each finding: reproduce → fix → prove the test fails without the fix
├─ reply to every finding, including rejections
├─ verify, amend, push
├─ verify, commit, push if authorized
├─ append what was settled to the prompt
└─ stop iff every reviewer emitted the stop token on the SAME commit
└─ stop iff every reviewer succeeded and emitted the stop token on the SAME commit
```

## Running reviewers

Cut a detached worktree at the pushed commit so reviewers see what a human reviewer would:
Resolve the requested PR's head to `REVIEW_SHA`, then cut a detached worktree at that exact
commit. After local fixes, review their new commit even when pushing is disabled:

```bash
git worktree add "$WT" origin/<branch> --detach
git worktree add "$WT" "$REVIEW_SHA" --detach
```

Launch concurrently and in the background — the slowest sets the round's wall clock. Agent
invocations are configuration (`config.example.yaml`); only three things vary:
Launch concurrently and in the background — the slowest sets the round's wall clock. Use the
current [agent definitions](https://github.com/agentsdance/codejury/tree/master/lib/agents) and
[configuration guide](https://github.com/agentsdance/codejury/blob/master/docs/configuration.md),
with overrides in `jury.config.json`
([example](https://github.com/agentsdance/codejury/blob/master/jury.config.example.json)):

- **prompt delivery** — argv (`codex`) vs a file flag (`droid -f`)
- **cwd** — process cwd (`codex`) vs an explicit `--cwd` flag (`droid`)
- **resume** — `codex exec resume --last` keeps context; droid's text output carries no session id, so
a follow-up must be a fresh run with its prior findings quoted back
- **resume** — retain each reviewer's exact session id and pass it to that agent's configured
resume command. Never select the most recent session: it may belong to another task. If there is
no usable session id, start fresh with only that reviewer's prior findings quoted back.

## The prompt

Four sections, in order. Template in `prompts/review-round.md`.
Four sections, in order. Template in [prompts/review-round.md](https://github.com/agentsdance/codejury/blob/master/prompts/review-round.md).

1. **What the change does** — a few bullets, so the reviewer does not infer intent.
2. **ALREADY SETTLED — do NOT re-report** — grows every round. Without it, each fresh reviewer
Expand Down Expand Up @@ -90,7 +103,7 @@ without reproducing anything, which is the whole point of the loop.

Reply to every finding — silence is not a resolution. Per finding:
**ACCEPTED** / **AGREE-BUT-DEFERRED** / **REJECTED**, each with reasoning, each ending in a direct
question. Template in `prompts/feedback.md`.
question. Template in [prompts/feedback.md](https://github.com/agentsdance/codejury/blob/master/prompts/feedback.md).

```bash
jury reply --dir "$WT" # one conversation per reviewer, concurrently
Expand All @@ -102,8 +115,9 @@ agreement stops being evidence. `jury reply` redacts the other agents' names out
for you, so writing "agy raised this too" in a `--reason` is safe — but do not rely on it as licence
to quote another thread.

Delivery follows `resume.supported`: codex resumes the session that already holds its review, so it
gets your verdicts alone; agy starts fresh every run, so its own findings are quoted back to it.
Delivery follows `resume.supported` and the recorded session id. Codex, Claude, Droid and
Antigravity support resuming their own conversations. Without a usable session id, start a fresh
conversation and quote only that reviewer's findings back to it.

Rejections need evidence, not authority. One was settled by writing a three-line throwaway test and
pasting the panic.
Expand All @@ -121,6 +135,7 @@ that is a design decision, not a fact, and the loop should stop arguing.
- **Mock frameworks may not re-apply inside a loop.** Build the mock once outside it.
- **A verbose agent's stdout contains the prompt**, so grepping the whole stream for the stop token
matches the instruction that asked for it. Parse the final report block.
- **A failed reviewer is not a sign-off**, even if its partial output includes the stop token.
- **Reviewers recycle suggestions when they have nothing left.** A verbatim repeat of a prior cosmetic
note is a decent convergence signal.

Expand Down
79 changes: 79 additions & 0 deletions plugin/skills/review/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
name: review
description: Run a Code Jury review of a pull request — independent AI reviewers report findings, one judge triages and fixes them, and the loop repeats until the reviewers agree there is nothing new.
argument-hint: "<pr-url> [pr-url ...] [jury flags]"
disable-model-invocation: true
---

# Code Jury review

Run the Code Jury CLI on: `$ARGUMENTS`

## 1. Require a PR URL

This command supports PR reviews only. Require at least one PR URL, supplied as a positional
argument or with `--pr`. If none is provided, reply briefly: "Please provide the PR URL to review,
for example:" followed by:

```text
/codejury:review https://github.com/owner/repo/pull/123
```

Wait for the URL before finding or launching the CLI. Do not infer a PR from the current branch
or start a local review. Flags alone, including `--dir` or `--resume`, do not supply a review target.

## 2. Find the CLI

Use `jury` when `command -v jury` finds it. Otherwise run it without installing, as
`npx -y @agentsdance/codejury`. It needs Node.js 20.10 or newer and npm; if neither works, explain
which prerequisite is missing and how to install it. Use the same CLI prefix for every command
below, including `agents`, `runs`, and reopening the console.

## 3. Build the command

Start from `jury review` (or `npx -y @agentsdance/codejury review`), then preserve the user's
arguments and quote each value safely for the shell. Do not evaluate argument text as shell code.

- One or more PR URLs select the PRs to review. Let Jury resolve their checkouts. With a PR URL,
`--dir` selects the checkout/state root. Resuming a run still requires its original PR URL(s).
- Add `--web=true` unless the arguments already set `--web`. Open the browser console by default;
an explicit `--web=false` or `--web false` disables it.
- Add `--push=false` unless the arguments already set `--push`. Jury pushes fixes by default; pushing
from inside this session should be the user's explicit choice. Say that you added it.

Recognize both `--push=false` and `--push false`, as well as bare `--push` or `--web` (true).
For example, preserve `--push false` and `--web=false` without appending conflicting defaults.
Explain that `--push=false` still lets the judge edit files and Jury create local commits.

Do not choose reviewers, a judge or models for the user; Jury already applies the flags, the
repository's `jury.config.json` and the saved defaults (`jury agents`).

## 4. Run it

Print the command, then run it with the Bash tool in the background: a review runs several agent
CLIs for several rounds and can take many minutes. Before agents start, Jury prints the selected
judge and reviewers; report them. Read the initial output until the console URL appears, then
share that exact URL as a clickable Markdown link, including its port and run query parameter.
Do not guess the URL. Skip this step when the user disabled the console. If it stops early (a
missing or unauthenticated agent CLI, a missing `gh` login for a GitHub URL), show the error and
the fix it names; `jury agents` lists which agent CLIs are installed.

Monitor the background output while the review runs. With the console enabled, do not rely on a
background-task completion notification: the server keeps the process alive after review ends.

## 5. Report

When Jury announces review completion or a stopped/incomplete review, summarize from the output:
the rounds run, each reviewer's findings with the judge's verdict (accepted, deferred or rejected),
the commits the judge made, and whether they were pushed. Distinguish convergence from a failed,
interrupted, or round-limited run. Report the result without waiting for the console to exit,
include its link again, and leave it running so the user can inspect the review. Do not stop the
console unless the user asks.

Every run is saved locally: `jury runs` lists them, and `jury --web-only` reopens the console to read
the full prompts, reports and verdicts. Retain any `--dir` used for the review in these commands and
select the reported run with `--run` when opening the console. Single-PR checkouts are temporary;
their unpushed fixes are removed during cleanup. Do not promise that those commits remain available.

Do not fix the findings yourself, and do not edit the files Jury is working on while it runs:
the judge owns the fixes for this review.
46 changes: 46 additions & 0 deletions test/plugin.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { existsSync, readdirSync, readFileSync } from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { execFileSync } from "node:child_process";

const root = fileURLToPath(new URL("..", import.meta.url));
const json = file => JSON.parse(readFileSync(path.join(root, file), "utf8"));

test("the marketplace lists the plugin under the same name as its manifest", () => {
const market = json(".claude-plugin/marketplace.json");
assert.equal(market.name, "codejury");
assert.ok(market.owner?.name);
const [entry] = market.plugins;
assert.equal(market.plugins.length, 1);
assert.match(entry.source, /^\.\/[^.]/);
const manifest = json(path.join(entry.source, ".claude-plugin/plugin.json"));
assert.equal(entry.name, manifest.name);
assert.equal(manifest.license, json("package.json").license);
// Unpinned on purpose: installs follow the repository instead of a version to keep in sync.
assert.equal(manifest.version, undefined);
// A plugin bin/ goes on PATH and blocks claude.ai installs; the repository's bin/ must stay outside.
assert.ok(!existsSync(path.join(root, entry.source, "bin")));
});

test("every plugin skill has a name and description, and the review skill uses real flags", () => {
const skills = path.join(root, "plugin/skills");
const names = readdirSync(skills).sort();
assert.deepEqual(names, ["multi-agent-code-review", "review"]);
for (const name of names) {
const text = readFileSync(path.join(skills, name, "SKILL.md"), "utf8");
const front = text.match(/^---\n([\s\S]*?)\n---\n/)?.[1];
assert.ok(front, `${name}: frontmatter`);
assert.match(front, new RegExp(`^name: ${name}$`, "m"));
assert.match(front, /^description: \S/m);
// Installed plugins are copied without the rest of the repository.
assert.doesNotMatch(text, /`(DESIGN\.md|config\.example\.yaml|prompts\/[\w-]+\.md)`/, `${name}: repository-relative path`);
}
const review = readFileSync(path.join(skills, "review/SKILL.md"), "utf8");
assert.match(review, /^disable-model-invocation: true$/m);
const help = execFileSync(process.execPath, [path.join(root, "bin/jury.js"), "help", "review"], { encoding: "utf8" });
for (const flag of review.match(/--[a-z][\w-]*/g)) assert.ok(help.includes(`${flag} `), `jury review has no ${flag}`);
assert.equal(json("package.json").name, "@agentsdance/codejury");
assert.match(review, /npx -y @agentsdance\/codejury/);
});
Loading