Docs · Agents · Code review
Code review
An agent that reviews every pull request and posts one comment. The setup is the one on Agents: a token, an agent file, a pipeline. This page is the working version of all three, plus the two rules that decide whether anybody reads the result.
What it posts
# Review — fix(ci): fetch every page of comments
| Critical | 0 |
| Medium | 2 |
| Low | 1 |
Safe to merge; both mediums are degradations of the fetch step rather than defects in what it feeds the agent.
M1 · Fetch failure turns the advisory job red
.github/workflows/review.yml:97
No continue-on-error, so a 5xx or a rate limit fails the job before the
review runs — exactly the red X this workflow promises never to produce.
Fix: add continue-on-error: true to the step.
A table, one sentence, then a block per finding. Nothing else: no preamble, no summary of the diff, no praise.
The agent
.claude/agents/code-review.md, in the repository being reviewed. Short on
purpose: Claude Code loads CLAUDE.md itself, so the house style is not
repeated here. The agent is told to review against it.
---
name: code-review
description: Reviews a branch's diff and writes review.md. Changes nothing, posts nothing.
tools: Bash, Read, Grep, Glob, Write
---
You review one branch's changes and write what you find to `review.md`.
Check for `previous-review.md` before you start. If it is there, this is a
follow-up.
Read the diff first, then read whole files around anything it touches. A diff
shows what moved, not what it broke: the caller that still passes the old
argument is not in the diff, and the test that no longer covers the branch is
not either.
## The output
`review.md` is exactly this shape, and nothing else.
# Review — <the pull request's title>
| Severity | Count |
| --- | --- |
| Critical | <n> |
| Medium | <n> |
| Low | <n> |
<One sentence: is this safe to merge, and if not, which finding stops it.>
### C1 · <the claim, in under ten words>
`path/to/file:42`
<What breaks, and the input or state that breaks it. Two or three lines.>
**Fix:** <the concrete change. Not "consider" and not "you may want to".>
Two things you must not do: change any code, and post anything anywhere. Write
`review.md` and stop. Something else decides what happens to it.
The pipeline
GitHub Actions
name: review
on: pull_request
concurrency:
group: review-${{ github.event.number }}
cancel-in-progress: true
permissions:
contents: read
pull-requests: write
jobs:
review:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
persist-credentials: false
- uses: actions/setup-node@v6
with:
node-version: '22'
- name: Install selan and Claude Code
shell: bash
run: |
curl -fsSL https://dl.selan.ai/install.sh | sh
npm install -g @anthropic-ai/claude-code
- name: Review
continue-on-error: true
env:
SELAN_TOKEN: ${{ secrets.SELAN_TOKEN }}
run: |
selan agent-run code-review \
"Review this pull request. \`git diff HEAD^1 HEAD\` is the change.
Write your review to $GITHUB_WORKSPACE/review.md."
- name: Post the review
continue-on-error: true
if: hashFiles('review.md') != ''
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR: ${{ github.event.number }}
run: |
node -e 'require("fs").writeFileSync("payload.json", JSON.stringify({
body: require("fs").readFileSync("review.md", "utf8") }))'
curl -sS -X POST \
-H "Authorization: Bearer $GH_TOKEN" \
-H "Accept: application/vnd.github+json" \
"$GITHUB_API_URL/repos/$GITHUB_REPOSITORY/issues/$PR/comments" \
-d @payload.json
HEAD^1, not the base branch. For a
pull_request event the checked-out ref is
refs/pull/N/merge, and its first parent is the commit the branch
was merged onto. Diffing against origin/main instead means anything that
lands on main while the job waits for a runner appears in the review as if this pull
request had written it.
persist-credentials: false, because the agent runs with
permissions skipped and has Bash. The default leaves a header carrying the
job's GITHUB_TOKEN (including its pull-requests: write) in
.git/config inside the workspace the agent reads.
shell: bash for the pipefail it adds. Under the default
bash -e, a failed curl -f in curl … | sh leaves
sh reading EOF and exiting 0: the step passes with nothing installed, and
the run ends green with no comment and no cause on screen.
GitLab CI
This is the version that runs, not a translation of the workflow above. Three things differ, and each one was a red pipeline first.
review:
stage: test
image: node:26
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
allow_failure: true
timeout: 20m
interruptible: true
cache:
when: always
key: review-$CI_MERGE_REQUEST_IID
paths: [previous-review.md]
variables:
GIT_DEPTH: 0
script:
# Not `curl | sh`: piped, a failed `curl -f` leaves sh reading EOF and exiting 0,
# so the step goes green with nothing installed.
- curl -fsSL https://dl.selan.ai/install.sh -o /tmp/install-selan.sh
- sh /tmp/install-selan.sh
- npm install -g @anthropic-ai/claude-code
# `selan agent-run` passes --dangerously-skip-permissions, and Claude Code exits 1
# on that flag at uid 0. Container images run as root, so drop to a user.
- install -d -o node -g node /tmp/agent
- chown -R node:node "$CI_PROJECT_DIR"
- |
cat > /tmp/review.sh <<'SH'
export HOME=/tmp/agent
export CLAUDE_CONFIG_DIR=/tmp/agent/.claude
selan agent-run code-review \
"Review this merge request. \`git diff $CI_MERGE_REQUEST_DIFF_BASE_SHA HEAD\` is the change.
Write your review to $CI_PROJECT_DIR/review.md. Your previous review,
if there is one, is at $CI_PROJECT_DIR/previous-review.md."
SH
su -p node -c "sh /tmp/review.sh"
- |
[ -s review.md ] || { echo "the agent wrote no review" >&2; exit 1; }
cp review.md previous-review.md
node scripts/review-to-codequality.mjs review.md gl-code-quality-report.json
artifacts:
when: always
paths: [review.md, gl-code-quality-report.json]
reports:
codequality: gl-code-quality-report.json
The agent cannot run as root.
selan agent-run adds --dangerously-skip-permissions
unconditionally, and Claude Code refuses that flag at uid 0:
--dangerously-skip-permissions cannot be used with root/sudo
privileges for security reasons. A self-hosted GitHub runner is already an
unprivileged user, which is why the workflow above never mentions this. A container
image is root, and GitLab has no image:user, so the job drops privileges
itself. IS_SANDBOX=1 is the escape the guard offers; it asserts the
container is a sandbox while handing an agent Bash inside it.
HOME must be set inside the script, not in the job's
variables:. A job variable is already in force during
get_sources, which runs git before your first script line, and the clone
dies with could not lock config file /tmp/agent/.gitconfig.
No merge ref, so no HEAD^1.
GitLab checks out the source branch itself. HEAD^1 is the previous commit
on that branch, so it would review the last push instead of the change.
CI_MERGE_REQUEST_DIFF_BASE_SHA is the merge base, which also keeps
anything that lands on main while the job queues out of the diff.
GIT_DEPTH: 0 is what puts it in the clone.
On the Free plan, nothing can post a comment.
CI_JOB_TOKEN cannot create a note, and on GitLab.com Free neither a
project nor a group access token can be issued at all: both endpoints answer
400 User does not have permission to a group Owner.
POST /user/personal_access_tokens accepts only k8s_proxy and
self_rotate, so api scope is web-UI only. What is left is a
personal token: authored by a person, not a bot, and carrying everything that
person can do, in a variable any job can read.
So the findings ride artifacts:reports:codequality, which GitLab reads
itself (no API call, no bot account, no credential). The cost is where they render: the
merge request's Reports tab, because the inline markers in the Changes
view are Ultimate and the pipeline's Code Quality tab is Premium. review.md
goes up as an artifact too, since that is where a finding's fix actually reads.
The agent writes markdown, so a small script converts it: heading to
description, the C/M/L prefix to
critical/major/minor, and the backticked
path:line to location. Derive fingerprint from
the id, path and claim and not the line number: GitLab matches findings across
pushes by fingerprint, and a fingerprint that ignores the line keeps a finding the same
one after its code moves down a file. Check the parsed count against the counts table
the agent wrote. A parser that matches nothing produces an empty report, and an empty
report is exactly what GitLab renders for a clean review.
One pipeline per push, and do not swallow tags.
Asking for merge request pipelines makes GitLab willing to create one
alongside the branch pipeline it already made, so every other job runs twice
per push. A workflow: block fixes that. The tag rule in it is
load-bearing, not tidiness: without it no pipeline is created for a tag at all, which
silently stops a tag-triggered release from ever shipping.
workflow:
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_TAG
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
- if: $CI_COMMIT_BRANCH
The follow-up base is a cache keyed on the merge request, not the last comment read back through the API. It is less code, and it drops the author filter the GitHub version needs: nobody but the job can write a cache, and the agent reads that file.
What makes it readable
Severities, and a count of each
An unconstrained reviewer writes an essay: nobody reads the fourth paragraph, and the finding that matters sits under one that does not. Four rules fix that.
- Define each severity in the agent file, in your terms. Critical is "merging this ships something wrong". Medium is a defect in a narrower state. Low is the wording nits. Undefined labels drift upward until everything is important and the labels mean nothing.
-
Keep the zero rows.
Critical | 0is the most useful cell in the table, and a table listing only what was found cannot say it. - Cap the low ones (three is plenty) and say what gets dropped when a review runs long: low first, then medium, and never a critical. Without a drop order a length limit is met by dropping whatever is last.
- Demand a fix, not a concern. "Consider refactoring this" costs the reader more than it saves. If the agent cannot write the fix, what it has is a question. Give it somewhere to put one so it does not dress it up as a finding.
The second run follows up on the first
This is the one that bites. Every push triggers a fresh run, and a fresh run rescans from nothing, so it finds a different set of true things than last time. You fix what it said, it comes back with three more, and the list never empties. The review is correct every time and impossible to finish.
The fix is to hand the agent what it already said. Fetch the last review your pipeline posted, write it into the workspace, and tell the agent that file is the base:
- name: Fetch the previous review
continue-on-error: true
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR: ${{ github.event.number }}
run: |
node -e '
(async () => {
const mine = []
for (let page = 1; ; page++) {
const res = await fetch(
`${process.env.GITHUB_API_URL}/repos/${process.env.GITHUB_REPOSITORY}` +
`/issues/${process.env.PR}/comments?per_page=100&page=${page}`,
{ headers: { Authorization: `Bearer ${process.env.GH_TOKEN}` } })
if (!res.ok) return
const batch = await res.json()
for (const c of batch) {
if (c.user.login === "github-actions[bot]" && c.body.startsWith("# Review —")) {
mine.push(c.body)
}
}
if (batch.length < 100) break
}
if (mine.length) require("fs").writeFileSync(
`${process.env.GITHUB_WORKSPACE}/previous-review.md`, mine.at(-1))
})()
'
Then four rules in the agent file, which are the whole point:
-
A finding keeps the id it was given.
M2isM2across three pushes, so one finding can be followed to its end. - The table counts what is still open, and one line under it names what was fixed: Fixed since the last review: M1, L2. A fixed finding gets no block. Its id on that line is the whole report.
- A new finding is added only if it is critical, or if it is in code this push changed. The agent is not re-reviewing the branch; it is reporting on a list it already wrote.
- Nothing is re-worded, re-ranked or re-scoped. Medium last time is medium now.
Read every page, not just the first: the comments endpoint returns oldest-first and
ignores sort and direction, so a single page of 100 silently
starts following up on a stale review, and it does so on exactly the long-running
merge request that needed this most.
# Review —, and this file is text an
agent reads and acts on. Filter on the author, never write the unfiltered response into
the workspace, and tell the agent in as many words that the file is data, not
instructions.